2026-02-25T08:03:31.849 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-25T08:03:31.863 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T08:03:31.940 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053 branch: wip-logstoremanager description: crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/admin_socket_objecter_requests} email: null first_in_suite: false flavor: debug job_id: '69053' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-logstoremanager ceph: conf: client.0: admin socket: /var/run/ceph/ceph-$name.asok global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 osd pool default crimson: true paxos service trim min: 10 osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd max markdown count: 1000 osd objectstore: seastore osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 osd target transaction size: 5 seastore cachepin size pershard: 64M seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - Degraded data redundancy - is down - osds down - pg .*? is .*?degraded.*?, acting - pg .*? is stuck - pg degraded - PG_BACKFILL_FULL - Low space hindering backfill .*? backfill_toofull - OSD_ROOT_DOWN - pgs degraded - pgs undersized - is active.*backfill_toofull.* sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: debug sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 thrashosds: min_in: 4 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: c32469671c6b0222f786dd36c34eb702f780a11d owner: scheduled_matan@soko04.front.sepia.ceph.com priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 869 sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: c32469671c6b0222f786dd36c34eb702f780a11d targets: trial042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHMcSn1dMgsIPlpZvacO3/oO2wo8fm3GXxt2upgM4Q2NaRvqxyoADLecnIfr8kxOpLNO6EONXNz4kJMRZREbL9w= trial066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJTyCjD9GrtQ38KMvpQXYdn4Q/0/aBehuDcfC5KMu3ylLZof83uU8dNjnHr3v4l3qEAERHjqK7Bv0sIjAbDs7Y= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: ceph_objectstore_tool: false chance_inject_pause_short: 0 chance_reset_purged_snaps_last: 0 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 chance_trim_stale_osdmaps: 0 dump_ops_enable: false min_in: 3 noscrub_toggle_delay: 0 reweight_osd: 0 sighup_delay: 0 thrash_primary_affinity: false timeout: 2400 - radosbench: clients: - client.0 time: 150 - admin_socket: client.0: objecter_requests: test: http://git.ceph.com/?p={repo};a=blob_plain;f=src/test/admin_socket/objecter_requests;hb={branch} teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-25_07:55:35 tube: trial user: matan verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-25T08:03:31.940 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa; will attempt to use it 2026-02-25T08:03:31.940 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks 2026-02-25T08:03:31.941 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-25T08:03:31.942 INFO:teuthology.task.internal:Checking packages... 2026-02-25T08:03:31.942 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '6aeb6975bc9a4790a399c9cc70dba9635d2a7819' 2026-02-25T08:03:31.942 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-25T08:03:31.942 INFO:teuthology.packaging:ref: None 2026-02-25T08:03:31.942 INFO:teuthology.packaging:tag: None 2026-02-25T08:03:31.942 INFO:teuthology.packaging:branch: wip-logstoremanager 2026-02-25T08:03:31.942 INFO:teuthology.packaging:sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:03:31.942 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-logstoremanager 2026-02-25T08:03:32.070 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5338.g6aeb6975 2026-02-25T08:03:32.071 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-25T08:03:32.083 INFO:teuthology.task.internal:no buildpackages task found 2026-02-25T08:03:32.083 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-25T08:03:32.101 INFO:teuthology.task.internal:Saving configuration 2026-02-25T08:03:32.115 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-25T08:03:32.127 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-25T08:03:32.204 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial042.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 07:57:37.140265', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHMcSn1dMgsIPlpZvacO3/oO2wo8fm3GXxt2upgM4Q2NaRvqxyoADLecnIfr8kxOpLNO6EONXNz4kJMRZREbL9w='} 2026-02-25T08:03:32.259 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial066.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 07:57:37.140876', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJTyCjD9GrtQ38KMvpQXYdn4Q/0/aBehuDcfC5KMu3ylLZof83uU8dNjnHr3v4l3qEAERHjqK7Bv0sIjAbDs7Y='} 2026-02-25T08:03:32.259 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-25T08:03:32.271 INFO:teuthology.task.internal:roles: ubuntu@trial042.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2026-02-25T08:03:32.271 INFO:teuthology.task.internal:roles: ubuntu@trial066.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2026-02-25T08:03:32.271 INFO:teuthology.run_tasks:Running task console_log... 2026-02-25T08:03:32.405 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7c8bce42f880>, signals=[15]) 2026-02-25T08:03:32.405 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-25T08:03:32.417 INFO:teuthology.task.internal:Opening connections... 2026-02-25T08:03:32.417 DEBUG:teuthology.task.internal:connecting to ubuntu@trial042.front.sepia.ceph.com 2026-02-25T08:03:32.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:03:32.482 DEBUG:teuthology.task.internal:connecting to ubuntu@trial066.front.sepia.ceph.com 2026-02-25T08:03:32.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:03:32.545 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-25T08:03:32.559 DEBUG:teuthology.orchestra.run.trial042:> uname -m 2026-02-25T08:03:32.576 INFO:teuthology.orchestra.run.trial042.stdout:x86_64 2026-02-25T08:03:32.576 DEBUG:teuthology.orchestra.run.trial042:> cat /etc/os-release 2026-02-25T08:03:32.631 INFO:teuthology.orchestra.run.trial042.stdout:NAME="CentOS Stream" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:VERSION="9" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:ID="centos" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:ID_LIKE="rhel fedora" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:VERSION_ID="9" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:PLATFORM_ID="platform:el9" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:ANSI_COLOR="0;31" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:LOGO="fedora-logo-icon" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T08:03:32.632 INFO:teuthology.orchestra.run.trial042.stdout:HOME_URL="https://centos.org/" 2026-02-25T08:03:32.633 INFO:teuthology.orchestra.run.trial042.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T08:03:32.633 INFO:teuthology.orchestra.run.trial042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T08:03:32.633 INFO:teuthology.orchestra.run.trial042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T08:03:32.634 INFO:teuthology.lock.ops:Updating trial042.front.sepia.ceph.com on lock server 2026-02-25T08:03:32.709 DEBUG:teuthology.orchestra.run.trial066:> uname -m 2026-02-25T08:03:32.726 INFO:teuthology.orchestra.run.trial066.stdout:x86_64 2026-02-25T08:03:32.726 DEBUG:teuthology.orchestra.run.trial066:> cat /etc/os-release 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:NAME="CentOS Stream" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:VERSION="9" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:ID="centos" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:ID_LIKE="rhel fedora" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:VERSION_ID="9" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:PLATFORM_ID="platform:el9" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:ANSI_COLOR="0;31" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:LOGO="fedora-logo-icon" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:HOME_URL="https://centos.org/" 2026-02-25T08:03:32.783 INFO:teuthology.orchestra.run.trial066.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T08:03:32.784 INFO:teuthology.orchestra.run.trial066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T08:03:32.784 INFO:teuthology.orchestra.run.trial066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T08:03:32.784 INFO:teuthology.lock.ops:Updating trial066.front.sepia.ceph.com on lock server 2026-02-25T08:03:32.870 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-25T08:03:32.883 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-25T08:03:32.895 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-25T08:03:32.895 DEBUG:teuthology.orchestra.run.trial042:> test '!' -e /home/ubuntu/cephtest 2026-02-25T08:03:32.898 DEBUG:teuthology.orchestra.run.trial066:> test '!' -e /home/ubuntu/cephtest 2026-02-25T08:03:32.913 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-25T08:03:32.926 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-25T08:03:32.927 DEBUG:teuthology.orchestra.run.trial042:> test -z $(ls -A /var/lib/ceph) 2026-02-25T08:03:32.954 DEBUG:teuthology.orchestra.run.trial066:> test -z $(ls -A /var/lib/ceph) 2026-02-25T08:03:32.971 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-25T08:03:33.138 INFO:teuthology.run_tasks:Running task kernel... 2026-02-25T08:03:33.156 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-25T08:03:33.156 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-25T08:03:33.156 DEBUG:teuthology.orchestra.run.trial042:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T08:03:33.157 DEBUG:teuthology.orchestra.run.trial066:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T08:03:33.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:03:33.171 DEBUG:teuthology.orchestra.run.trial066:> uname -r 2026-02-25T08:03:33.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:03:33.172 DEBUG:teuthology.orchestra.run.trial042:> uname -r 2026-02-25T08:03:33.226 INFO:teuthology.orchestra.run.trial066.stdout:5.14.0-681.el9.x86_64 2026-02-25T08:03:33.226 INFO:teuthology.task.kernel:Running kernel on trial066: 5.14.0-681.el9.x86_64 2026-02-25T08:03:33.227 DEBUG:teuthology.orchestra.run.trial066:> sudo yum install -y kernel 2026-02-25T08:03:33.228 INFO:teuthology.orchestra.run.trial042.stdout:5.14.0-681.el9.x86_64 2026-02-25T08:03:33.228 INFO:teuthology.task.kernel:Running kernel on trial042: 5.14.0-681.el9.x86_64 2026-02-25T08:03:33.228 DEBUG:teuthology.orchestra.run.trial042:> sudo yum install -y kernel 2026-02-25T08:03:34.601 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.9 MB 00:00 2026-02-25T08:03:35.545 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.9 MB 00:01 2026-02-25T08:03:37.789 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-25T08:03:38.737 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-25T08:03:42.155 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 8.0 MB 00:01 2026-02-25T08:03:42.669 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 8.0 MB 00:02 2026-02-25T08:03:43.567 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - Extras packages 126 kB/s | 20 kB 00:00 2026-02-25T08:03:44.150 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 20 kB 00:01 2026-02-25T08:03:44.388 INFO:teuthology.orchestra.run.trial066.stdout:Extra Packages for Enterprise Linux 27 MB/s | 20 MB 00:00 2026-02-25T08:03:45.025 INFO:teuthology.orchestra.run.trial042.stdout:Extra Packages for Enterprise Linux 25 MB/s | 20 MB 00:00 2026-02-25T08:03:47.531 INFO:teuthology.orchestra.run.trial066.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-25T08:03:48.216 INFO:teuthology.orchestra.run.trial042.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T08:03:48.539 INFO:teuthology.orchestra.run.trial066.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T08:03:48.539 INFO:teuthology.orchestra.run.trial066.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T08:03:48.560 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:03:48.561 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:03:48.562 INFO:teuthology.orchestra.run.trial066.stdout: Package Architecture Version Repository Size 2026-02-25T08:03:48.562 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:03:48.562 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:03:48.562 INFO:teuthology.orchestra.run.trial066.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T08:03:48.562 INFO:teuthology.orchestra.run.trial066.stdout:Installing dependencies: 2026-02-25T08:03:48.563 INFO:teuthology.orchestra.run.trial066.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T08:03:48.563 INFO:teuthology.orchestra.run.trial066.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T08:03:48.563 INFO:teuthology.orchestra.run.trial066.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T08:03:48.563 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:03:48.563 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:03:48.563 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:03:48.564 INFO:teuthology.orchestra.run.trial066.stdout:Install 4 Packages 2026-02-25T08:03:48.564 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:03:48.564 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 89 M 2026-02-25T08:03:48.564 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 135 M 2026-02-25T08:03:48.564 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:03:48.815 INFO:teuthology.orchestra.run.trial066.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 5.0 MB/s | 901 kB 00:00 2026-02-25T08:03:49.063 INFO:teuthology.orchestra.run.trial066.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 41 MB/s | 17 MB 00:00 2026-02-25T08:03:49.214 INFO:teuthology.orchestra.run.trial066.stdout:(3/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 69 MB/s | 40 MB 00:00 2026-02-25T08:03:49.223 INFO:teuthology.orchestra.run.trial042.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T08:03:49.223 INFO:teuthology.orchestra.run.trial042.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T08:03:49.243 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout: Package Architecture Version Repository Size 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:Installing dependencies: 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:Install 4 Packages 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 89 M 2026-02-25T08:03:49.244 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 135 M 2026-02-25T08:03:49.245 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:03:49.282 INFO:teuthology.orchestra.run.trial066.stdout:(4/4): kernel-modules-core-5.14.0-686.el9.x86_6 66 MB/s | 31 MB 00:00 2026-02-25T08:03:49.283 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:03:49.283 INFO:teuthology.orchestra.run.trial066.stdout:Total 123 MB/s | 89 MB 00:00 2026-02-25T08:03:49.470 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:03:49.673 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:03:49.673 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:03:49.990 INFO:teuthology.orchestra.run.trial042.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 1.3 MB/s | 901 kB 00:00 2026-02-25T08:03:50.057 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:03:50.057 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:03:50.526 INFO:teuthology.orchestra.run.trial042.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 14 MB/s | 17 MB 00:01 2026-02-25T08:03:51.051 INFO:teuthology.orchestra.run.trial042.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 29 MB/s | 31 MB 00:01 2026-02-25T08:03:51.166 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:03:51.199 INFO:teuthology.orchestra.run.trial042.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 21 MB/s | 40 MB 00:01 2026-02-25T08:03:51.199 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:03:51.199 INFO:teuthology.orchestra.run.trial042.stdout:Total 46 MB/s | 89 MB 00:01 2026-02-25T08:03:51.392 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:03:51.454 INFO:teuthology.orchestra.run.trial066.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:03:51.476 INFO:teuthology.orchestra.run.trial066.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:03:51.591 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:03:51.591 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:03:51.972 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:03:51.973 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:03:53.079 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:03:53.369 INFO:teuthology.orchestra.run.trial042.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:03:53.391 INFO:teuthology.orchestra.run.trial042.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:03:54.047 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:03:54.081 INFO:teuthology.orchestra.run.trial066.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:03:55.941 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:03:55.975 INFO:teuthology.orchestra.run.trial042.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:03:57.182 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:03:57.189 INFO:teuthology.orchestra.run.trial066.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:03:59.017 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:03:59.024 INFO:teuthology.orchestra.run.trial042.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:00.296 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:02.138 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:17.726 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:17.748 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:17.937 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:17.937 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:04:17.937 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:04:17.937 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:04:17.995 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:04:18.057 DEBUG:teuthology.orchestra.run.trial066:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:04:18.088 INFO:teuthology.orchestra.run.trial066.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:04:18.088 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:04:18.089 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T08:04:18.089 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T08:04:18.089 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T08:04:18.089 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-25T08:04:18.089 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial066.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T08:04:18.089 DEBUG:teuthology.orchestra.run.trial066:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:04:18.153 INFO:teuthology.orchestra.run.trial066.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:04:18.153 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:04:18.154 DEBUG:teuthology.orchestra.run.trial066:> sudo rpm -qi grub2-tools 2026-02-25T08:04:18.222 INFO:teuthology.orchestra.run.trial066.stdout:Name : grub2-tools 2026-02-25T08:04:18.222 INFO:teuthology.orchestra.run.trial066.stdout:Epoch : 1 2026-02-25T08:04:18.223 INFO:teuthology.orchestra.run.trial066.stdout:Version : 2.06 2026-02-25T08:04:18.223 INFO:teuthology.orchestra.run.trial066.stdout:Release : 120.el9 2026-02-25T08:04:18.223 INFO:teuthology.orchestra.run.trial066.stdout:Architecture: x86_64 2026-02-25T08:04:18.223 INFO:teuthology.orchestra.run.trial066.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T08:04:18.223 INFO:teuthology.orchestra.run.trial066.stdout:Group : Unspecified 2026-02-25T08:04:18.223 INFO:teuthology.orchestra.run.trial066.stdout:Size : 9108611 2026-02-25T08:04:18.223 INFO:teuthology.orchestra.run.trial066.stdout:License : GPLv3+ 2026-02-25T08:04:18.223 INFO:teuthology.orchestra.run.trial066.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:Packager : builder@centos.org 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:Vendor : CentOS 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:Summary : Support tools for GRUB. 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:Description : 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T08:04:18.224 INFO:teuthology.orchestra.run.trial066.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T08:04:18.225 INFO:teuthology.orchestra.run.trial066.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T08:04:18.225 INFO:teuthology.orchestra.run.trial066.stdout:hardware devices. 2026-02-25T08:04:18.225 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:04:18.225 INFO:teuthology.orchestra.run.trial066.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T08:04:18.226 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T08:04:18.226 INFO:teuthology.task.kernel:Updating grub on trial066 to boot 5.14.0-686.el9.x86_64 2026-02-25T08:04:18.226 DEBUG:teuthology.orchestra.run.trial066:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T08:04:18.290 INFO:teuthology.orchestra.run.trial066.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T08:04:18.291 DEBUG:teuthology.orchestra.run.trial066:> sudo command -v grubby && echo yes || echo no 2026-02-25T08:04:18.354 INFO:teuthology.orchestra.run.trial066.stdout:/sbin/grubby 2026-02-25T08:04:18.355 INFO:teuthology.orchestra.run.trial066.stdout:yes 2026-02-25T08:04:18.355 DEBUG:teuthology.orchestra.run.trial066:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T08:04:18.418 INFO:teuthology.orchestra.run.trial066.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T08:04:18.420 DEBUG:teuthology.orchestra.run.trial066:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T08:04:18.492 DEBUG:teuthology.orchestra.run.trial066:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T08:04:18.698 INFO:teuthology.orchestra.run.trial066.stderr:Generating grub configuration file ... 2026-02-25T08:04:18.765 INFO:teuthology.orchestra.run.trial066.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T08:04:18.772 INFO:teuthology.orchestra.run.trial066.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T08:04:19.020 INFO:teuthology.orchestra.run.trial066.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T08:04:19.026 INFO:teuthology.orchestra.run.trial066.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T08:04:19.060 INFO:teuthology.orchestra.run.trial066.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T08:04:19.066 INFO:teuthology.orchestra.run.trial066.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T08:04:19.098 INFO:teuthology.orchestra.run.trial066.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T08:04:19.105 INFO:teuthology.orchestra.run.trial066.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T08:04:19.129 INFO:teuthology.orchestra.run.trial066.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T08:04:19.135 INFO:teuthology.orchestra.run.trial066.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T08:04:19.222 INFO:teuthology.orchestra.run.trial066.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T08:04:19.230 INFO:teuthology.orchestra.run.trial066.stderr:done 2026-02-25T08:04:19.232 DEBUG:teuthology.orchestra.run.trial066:> test -d /sys/firmware/efi 2026-02-25T08:04:19.248 DEBUG:teuthology.orchestra.run.trial066:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T08:04:19.313 INFO:teuthology.orchestra.run.trial066.stdout:/boot/efi/EFI/BOOT 2026-02-25T08:04:19.313 INFO:teuthology.orchestra.run.trial066.stdout:/boot/efi/EFI/centos 2026-02-25T08:04:19.315 DEBUG:teuthology.orchestra.run.trial066:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T08:04:19.378 INFO:teuthology.orchestra.run.trial066.stdout:no 2026-02-25T08:04:19.379 DEBUG:teuthology.orchestra.run.trial066:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T08:04:19.441 INFO:teuthology.orchestra.run.trial066.stdout:yes 2026-02-25T08:04:19.441 DEBUG:teuthology.orchestra.run.trial066:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T08:04:19.506 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial066 2026-02-25T08:04:19.506 DEBUG:teuthology.orchestra.run.trial066:> sudo bash -c ' 2026-02-25T08:04:19.506 DEBUG:teuthology.orchestra.run.trial066:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T08:04:19.506 DEBUG:teuthology.orchestra.run.trial066:> ' 2026-02-25T08:04:19.882 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:19.904 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:20.093 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:20.094 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:04:20.094 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:04:20.094 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:04:20.152 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:04:20.153 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:04:20.153 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:04:20.153 INFO:teuthology.orchestra.run.trial042.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T08:04:20.153 INFO:teuthology.orchestra.run.trial042.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T08:04:20.153 INFO:teuthology.orchestra.run.trial042.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T08:04:20.153 INFO:teuthology.orchestra.run.trial042.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T08:04:20.153 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:04:20.153 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:04:20.214 DEBUG:teuthology.orchestra.run.trial042:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:04:20.243 INFO:teuthology.orchestra.run.trial042.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:04:20.244 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:04:20.244 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T08:04:20.244 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T08:04:20.244 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T08:04:20.244 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-25T08:04:20.244 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial042.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T08:04:20.245 DEBUG:teuthology.orchestra.run.trial042:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:04:20.311 INFO:teuthology.orchestra.run.trial042.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:04:20.311 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:04:20.311 DEBUG:teuthology.orchestra.run.trial042:> sudo rpm -qi grub2-tools 2026-02-25T08:04:20.379 INFO:teuthology.orchestra.run.trial042.stdout:Name : grub2-tools 2026-02-25T08:04:20.379 INFO:teuthology.orchestra.run.trial042.stdout:Epoch : 1 2026-02-25T08:04:20.379 INFO:teuthology.orchestra.run.trial042.stdout:Version : 2.06 2026-02-25T08:04:20.379 INFO:teuthology.orchestra.run.trial042.stdout:Release : 120.el9 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Architecture: x86_64 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Group : Unspecified 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Size : 9108611 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:License : GPLv3+ 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Packager : builder@centos.org 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Vendor : CentOS 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Summary : Support tools for GRUB. 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:Description : 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T08:04:20.380 INFO:teuthology.orchestra.run.trial042.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T08:04:20.381 INFO:teuthology.orchestra.run.trial042.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T08:04:20.381 INFO:teuthology.orchestra.run.trial042.stdout:hardware devices. 2026-02-25T08:04:20.381 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:04:20.381 INFO:teuthology.orchestra.run.trial042.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T08:04:20.381 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T08:04:20.382 INFO:teuthology.task.kernel:Updating grub on trial042 to boot 5.14.0-686.el9.x86_64 2026-02-25T08:04:20.382 DEBUG:teuthology.orchestra.run.trial042:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T08:04:20.445 INFO:teuthology.orchestra.run.trial042.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T08:04:20.446 DEBUG:teuthology.orchestra.run.trial042:> sudo command -v grubby && echo yes || echo no 2026-02-25T08:04:20.509 INFO:teuthology.orchestra.run.trial042.stdout:/sbin/grubby 2026-02-25T08:04:20.510 INFO:teuthology.orchestra.run.trial042.stdout:yes 2026-02-25T08:04:20.510 DEBUG:teuthology.orchestra.run.trial042:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T08:04:20.575 INFO:teuthology.orchestra.run.trial042.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T08:04:20.576 DEBUG:teuthology.orchestra.run.trial042:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T08:04:20.649 DEBUG:teuthology.orchestra.run.trial042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T08:04:20.858 INFO:teuthology.orchestra.run.trial042.stderr:Generating grub configuration file ... 2026-02-25T08:04:20.925 INFO:teuthology.orchestra.run.trial042.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T08:04:20.932 INFO:teuthology.orchestra.run.trial042.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T08:04:21.184 INFO:teuthology.orchestra.run.trial042.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T08:04:21.191 INFO:teuthology.orchestra.run.trial042.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T08:04:21.226 INFO:teuthology.orchestra.run.trial042.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T08:04:21.232 INFO:teuthology.orchestra.run.trial042.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T08:04:21.263 INFO:teuthology.orchestra.run.trial042.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T08:04:21.269 INFO:teuthology.orchestra.run.trial042.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T08:04:21.292 INFO:teuthology.orchestra.run.trial042.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T08:04:21.298 INFO:teuthology.orchestra.run.trial042.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T08:04:21.384 INFO:teuthology.orchestra.run.trial042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T08:04:21.392 INFO:teuthology.orchestra.run.trial042.stderr:done 2026-02-25T08:04:21.394 DEBUG:teuthology.orchestra.run.trial042:> test -d /sys/firmware/efi 2026-02-25T08:04:21.410 DEBUG:teuthology.orchestra.run.trial042:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T08:04:21.475 INFO:teuthology.orchestra.run.trial042.stdout:/boot/efi/EFI/BOOT 2026-02-25T08:04:21.475 INFO:teuthology.orchestra.run.trial042.stdout:/boot/efi/EFI/centos 2026-02-25T08:04:21.476 DEBUG:teuthology.orchestra.run.trial042:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T08:04:21.541 INFO:teuthology.orchestra.run.trial042.stdout:no 2026-02-25T08:04:21.541 DEBUG:teuthology.orchestra.run.trial042:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T08:04:21.606 INFO:teuthology.orchestra.run.trial042.stdout:yes 2026-02-25T08:04:21.606 DEBUG:teuthology.orchestra.run.trial042:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T08:04:21.671 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial042 2026-02-25T08:04:21.671 DEBUG:teuthology.orchestra.run.trial042:> sudo bash -c ' 2026-02-25T08:04:21.671 DEBUG:teuthology.orchestra.run.trial042:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T08:04:21.671 DEBUG:teuthology.orchestra.run.trial042:> ' 2026-02-25T08:04:49.550 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-25T08:04:49.551 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-25T08:04:49.551 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:04:51.714 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-25T08:04:51.715 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial042.front.sepia.ceph.com' 2026-02-25T08:04:51.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:05:03.808 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.66 2026-02-25T08:05:05.983 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.42 2026-02-25T08:05:12.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-25T08:05:12.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:05:14.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial042.front.sepia.ceph.com' 2026-02-25T08:05:14.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:05:48.624 DEBUG:teuthology.orchestra.run.trial066:> true 2026-02-25T08:05:48.777 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-25T08:05:48.778 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-686.el9.x86_64"... 2026-02-25T08:05:48.778 DEBUG:teuthology.orchestra.run.trial066:> uname -r 2026-02-25T08:05:48.831 INFO:teuthology.orchestra.run.trial066.stdout:5.14.0-686.el9.x86_64 2026-02-25T08:05:48.831 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T08:05:48.831 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T08:05:48.831 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T08:05:49.832 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-25T08:05:49.832 DEBUG:teuthology.orchestra.run.trial066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T08:05:49.863 INFO:teuthology.orchestra.run.trial066.stdout:ttyS1 2026-02-25T08:05:49.882 DEBUG:teuthology.parallel:result is None 2026-02-25T08:05:51.195 DEBUG:teuthology.orchestra.run.trial042:> true 2026-02-25T08:05:51.351 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial042.front.sepia.ceph.com' 2026-02-25T08:05:51.351 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-25T08:05:51.352 DEBUG:teuthology.orchestra.run.trial042:> uname -r 2026-02-25T08:05:51.409 INFO:teuthology.orchestra.run.trial042.stdout:5.14.0-686.el9.x86_64 2026-02-25T08:05:51.409 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T08:05:51.409 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T08:05:51.409 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T08:05:52.410 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-25T08:05:52.410 DEBUG:teuthology.orchestra.run.trial042:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T08:05:52.440 INFO:teuthology.orchestra.run.trial042.stdout:ttyS1 2026-02-25T08:05:52.462 DEBUG:teuthology.parallel:result is None 2026-02-25T08:05:52.462 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-25T08:05:52.476 INFO:teuthology.task.internal:Creating test directory... 2026-02-25T08:05:52.476 DEBUG:teuthology.orchestra.run.trial042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T08:05:52.504 DEBUG:teuthology.orchestra.run.trial066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T08:05:52.521 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-25T08:05:52.536 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-25T08:05:52.550 INFO:teuthology.task.internal:Creating archive directory... 2026-02-25T08:05:52.550 DEBUG:teuthology.orchestra.run.trial042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T08:05:52.560 DEBUG:teuthology.orchestra.run.trial066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T08:05:52.580 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-25T08:05:52.591 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-25T08:05:52.591 DEBUG:teuthology.orchestra.run.trial042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T08:05:52.616 DEBUG:teuthology.orchestra.run.trial066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T08:05:52.638 INFO:teuthology.orchestra.run.trial042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:05:52.640 INFO:teuthology.orchestra.run.trial066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:05:52.646 INFO:teuthology.orchestra.run.trial042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:05:52.649 INFO:teuthology.orchestra.run.trial066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:05:52.651 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-25T08:05:52.662 INFO:teuthology.task.internal:Configuring sudo... 2026-02-25T08:05:52.662 DEBUG:teuthology.orchestra.run.trial042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T08:05:52.690 DEBUG:teuthology.orchestra.run.trial066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T08:05:52.713 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-25T08:05:52.726 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-25T08:05:52.726 DEBUG:teuthology.orchestra.run.trial042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T08:05:52.754 DEBUG:teuthology.orchestra.run.trial066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T08:05:52.769 DEBUG:teuthology.orchestra.run.trial042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:05:52.887 DEBUG:teuthology.orchestra.run.trial042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:05:52.912 DEBUG:teuthology.orchestra.run.trial042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:05:53.052 DEBUG:teuthology.orchestra.run.trial042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:05:53.076 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:05:53.076 DEBUG:teuthology.orchestra.run.trial042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T08:05:53.139 DEBUG:teuthology.orchestra.run.trial066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:05:53.217 DEBUG:teuthology.orchestra.run.trial066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:05:53.243 DEBUG:teuthology.orchestra.run.trial066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:05:53.359 DEBUG:teuthology.orchestra.run.trial066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:05:53.383 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:05:53.383 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T08:05:53.446 DEBUG:teuthology.orchestra.run.trial042:> sudo service rsyslog restart 2026-02-25T08:05:53.448 DEBUG:teuthology.orchestra.run.trial066:> sudo service rsyslog restart 2026-02-25T08:05:53.474 INFO:teuthology.orchestra.run.trial042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:05:53.514 INFO:teuthology.orchestra.run.trial066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:05:54.043 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-25T08:05:54.054 INFO:teuthology.task.internal:Starting timer... 2026-02-25T08:05:54.054 INFO:teuthology.run_tasks:Running task pcp... 2026-02-25T08:05:54.072 INFO:teuthology.run_tasks:Running task selinux... 2026-02-25T08:05:54.085 DEBUG:teuthology.orchestra.run.trial042:> sudo service auditd rotate 2026-02-25T08:05:54.115 INFO:teuthology.orchestra.run.trial042.stdout:Rotating logs: 2026-02-25T08:05:54.117 DEBUG:teuthology.orchestra.run.trial066:> sudo service auditd rotate 2026-02-25T08:05:54.146 INFO:teuthology.orchestra.run.trial066.stdout:Rotating logs: 2026-02-25T08:05:54.148 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-25T08:05:54.148 DEBUG:teuthology.orchestra.run.trial042:> /usr/sbin/getenforce 2026-02-25T08:05:54.173 INFO:teuthology.orchestra.run.trial042.stdout:Permissive 2026-02-25T08:05:54.173 DEBUG:teuthology.orchestra.run.trial066:> /usr/sbin/getenforce 2026-02-25T08:05:54.204 INFO:teuthology.orchestra.run.trial066.stdout:Permissive 2026-02-25T08:05:54.205 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial042.front.sepia.ceph.com': 'permissive', 'ubuntu@trial066.front.sepia.ceph.com': 'permissive'} 2026-02-25T08:05:54.205 DEBUG:teuthology.orchestra.run.trial042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:05:54.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:05:54.237 DEBUG:teuthology.orchestra.run.trial066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:05:54.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:05:54.269 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-25T08:05:54.269 DEBUG:teuthology.orchestra.run.trial042:> sudo /usr/sbin/setenforce permissive 2026-02-25T08:05:54.301 DEBUG:teuthology.orchestra.run.trial066:> sudo /usr/sbin/setenforce permissive 2026-02-25T08:05:54.332 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-25T08:05:54.345 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-25T08:05:54.351 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-25T08:05:54.351 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial042.front.sepia.ceph.com,trial066.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-25T08:08:23.970 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial042.front.sepia.ceph.com'), Remote(name='ubuntu@trial066.front.sepia.ceph.com')] 2026-02-25T08:08:23.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial042.front.sepia.ceph.com' 2026-02-25T08:08:23.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:08:24.078 DEBUG:teuthology.orchestra.run.trial042:> true 2026-02-25T08:08:24.154 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial042.front.sepia.ceph.com' 2026-02-25T08:08:24.154 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-25T08:08:24.154 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:08:24.256 DEBUG:teuthology.orchestra.run.trial066:> true 2026-02-25T08:08:24.327 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial066.front.sepia.ceph.com' 2026-02-25T08:08:24.327 INFO:teuthology.run_tasks:Running task clock... 2026-02-25T08:08:24.340 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-25T08:08:24.340 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T08:08:24.341 DEBUG:teuthology.orchestra.run.trial042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:08:24.343 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T08:08:24.343 DEBUG:teuthology.orchestra.run.trial066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:08:24.370 INFO:teuthology.orchestra.run.trial042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T08:08:24.381 INFO:teuthology.orchestra.run.trial042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T08:08:24.394 INFO:teuthology.orchestra.run.trial066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T08:08:24.400 INFO:teuthology.orchestra.run.trial042.stderr:sudo: ntpd: command not found 2026-02-25T08:08:24.406 INFO:teuthology.orchestra.run.trial066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T08:08:24.409 INFO:teuthology.orchestra.run.trial042.stdout:506 Cannot talk to daemon 2026-02-25T08:08:24.421 INFO:teuthology.orchestra.run.trial042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T08:08:24.427 INFO:teuthology.orchestra.run.trial066.stderr:sudo: ntpd: command not found 2026-02-25T08:08:24.432 INFO:teuthology.orchestra.run.trial042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T08:08:24.437 INFO:teuthology.orchestra.run.trial066.stdout:506 Cannot talk to daemon 2026-02-25T08:08:24.449 INFO:teuthology.orchestra.run.trial066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T08:08:24.461 INFO:teuthology.orchestra.run.trial066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T08:08:24.476 INFO:teuthology.orchestra.run.trial042.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:08:24.481 INFO:teuthology.orchestra.run.trial042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:08:24.481 INFO:teuthology.orchestra.run.trial042.stdout:=============================================================================== 2026-02-25T08:08:24.481 INFO:teuthology.orchestra.run.trial042.stdout:^? gwallaceNTPpool.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:08:24.508 INFO:teuthology.orchestra.run.trial066.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:08:24.515 INFO:teuthology.orchestra.run.trial066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:08:24.515 INFO:teuthology.orchestra.run.trial066.stdout:=============================================================================== 2026-02-25T08:08:24.515 INFO:teuthology.orchestra.run.trial066.stdout:^? time.cloudflare.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T08:08:24.516 INFO:teuthology.run_tasks:Running task install... 2026-02-25T08:08:24.527 DEBUG:teuthology.task.install:project ceph 2026-02-25T08:08:24.527 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}} 2026-02-25T08:08:24.527 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'} 2026-02-25T08:08:24.527 INFO:teuthology.task.install:Using flavor: debug 2026-02-25T08:08:24.530 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T08:08:24.530 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-25T08:08:24.530 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:08:24.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:08:24.533 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:08:24.533 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:08:24.728 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:08:24.728 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:08:24.739 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:08:24.739 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:08:25.083 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:08:25.083 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:08:25.083 DEBUG:teuthology.orchestra.run.trial042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:08:25.109 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-25T08:08:25.109 DEBUG:teuthology.orchestra.run.trial042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:08:25.136 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:08:25.136 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:08:25.136 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:08:25.162 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-25T08:08:25.162 DEBUG:teuthology.orchestra.run.trial066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:08:25.173 DEBUG:teuthology.orchestra.run.trial042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:08:25.227 DEBUG:teuthology.orchestra.run.trial066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:08:25.247 DEBUG:teuthology.orchestra.run.trial042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:08:25.299 DEBUG:teuthology.orchestra.run.trial066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:08:25.312 INFO:teuthology.orchestra.run.trial042.stdout:check_obsoletes = 1 2026-02-25T08:08:25.313 DEBUG:teuthology.orchestra.run.trial042:> sudo yum clean all 2026-02-25T08:08:25.363 INFO:teuthology.orchestra.run.trial066.stdout:check_obsoletes = 1 2026-02-25T08:08:25.365 DEBUG:teuthology.orchestra.run.trial066:> sudo yum clean all 2026-02-25T08:08:25.462 INFO:teuthology.orchestra.run.trial042.stdout:41 files removed 2026-02-25T08:08:25.474 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-radosgw 2026-02-25T08:08:25.515 INFO:teuthology.orchestra.run.trial066.stdout:41 files removed 2026-02-25T08:08:25.529 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-radosgw 2026-02-25T08:08:26.311 INFO:teuthology.orchestra.run.trial042.stdout:ceph packages for x86_64 136 kB/s | 94 kB 00:00 2026-02-25T08:08:26.386 INFO:teuthology.orchestra.run.trial066.stdout:ceph packages for x86_64 132 kB/s | 94 kB 00:00 2026-02-25T08:08:26.983 INFO:teuthology.orchestra.run.trial042.stdout:ceph noarch packages 38 kB/s | 24 kB 00:00 2026-02-25T08:08:27.066 INFO:teuthology.orchestra.run.trial066.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-25T08:08:27.544 INFO:teuthology.orchestra.run.trial042.stdout:ceph source packages 3.9 kB/s | 2.1 kB 00:00 2026-02-25T08:08:27.617 INFO:teuthology.orchestra.run.trial066.stdout:ceph source packages 3.9 kB/s | 2.1 kB 00:00 2026-02-25T08:08:28.395 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-25T08:08:28.672 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.9 MB 00:01 2026-02-25T08:08:31.913 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 27 MB 00:02 2026-02-25T08:08:32.186 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 27 MB 00:03 2026-02-25T08:08:43.220 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - CRB 915 kB/s | 8.0 MB 00:08 2026-02-25T08:08:43.834 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - CRB 884 kB/s | 8.0 MB 00:09 2026-02-25T08:08:44.143 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - Extras packages 112 kB/s | 20 kB 00:00 2026-02-25T08:08:44.751 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - Extras packages 126 kB/s | 20 kB 00:00 2026-02-25T08:08:45.695 INFO:teuthology.orchestra.run.trial042.stdout:Extra Packages for Enterprise Linux 23 MB/s | 20 MB 00:00 2026-02-25T08:08:46.213 INFO:teuthology.orchestra.run.trial066.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:02 2026-02-25T08:08:48.953 INFO:teuthology.orchestra.run.trial042.stdout:lab-extras 1.1 MB/s | 50 kB 00:00 2026-02-25T08:08:49.376 INFO:teuthology.orchestra.run.trial066.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T08:08:50.157 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repository Size 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: ceph-radosgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 11 M 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout:Upgrading: 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: librados2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 3.3 M 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: librbd1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.7 M 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout:Installing dependencies: 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: ceph-base x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.1 M 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 21 M 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: ceph-selinux x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 25 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: libcephfs2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 860 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: libradosstriper1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 263 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: librgw2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.6 M 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: python3-ceph-argparse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 45 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: python3-ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 185 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: python3-cephfs x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 204 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: python3-rados x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 324 k 2026-02-25T08:08:50.159 INFO:teuthology.orchestra.run.trial042.stdout: python3-rbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 306 k 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: python3-rgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 100 k 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout:Installing weak dependencies: 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: libcephfs-daemon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 26 k 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout:Install 35 Packages 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout:Upgrade 2 Packages 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 61 M 2026-02-25T08:08:50.160 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:08:50.594 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repository Size 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: ceph-radosgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 11 M 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout:Upgrading: 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: librados2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 3.3 M 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: librbd1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.7 M 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout:Installing dependencies: 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: ceph-base x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.1 M 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 21 M 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: ceph-selinux x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 25 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 860 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: libradosstriper1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 263 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: librgw2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.6 M 2026-02-25T08:08:50.596 INFO:teuthology.orchestra.run.trial066.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: python3-ceph-argparse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 45 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: python3-ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 185 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: python3-cephfs x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 204 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: python3-rados x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 324 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: python3-rbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 306 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: python3-rgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 100 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout:Installing weak dependencies: 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs-daemon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 26 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout:Install 35 Packages 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout:Upgrade 2 Packages 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 61 M 2026-02-25T08:08:50.597 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:08:54.719 INFO:teuthology.orchestra.run.trial066.stdout:(1/37): ceph-base-20.3.0-5338.g6aeb6975.el9.x86 1.4 MB/s | 5.1 MB 00:03 2026-02-25T08:08:54.791 INFO:teuthology.orchestra.run.trial066.stdout:(2/37): ceph-selinux-20.3.0-5338.g6aeb6975.el9. 350 kB/s | 25 kB 00:00 2026-02-25T08:08:54.863 INFO:teuthology.orchestra.run.trial066.stdout:(3/37): libcephfs-daemon-20.3.0-5338.g6aeb6975. 516 kB/s | 37 kB 00:00 2026-02-25T08:08:54.933 INFO:teuthology.orchestra.run.trial066.stdout:(4/37): libcephfs-proxy2-20.3.0-5338.g6aeb6975. 362 kB/s | 26 kB 00:00 2026-02-25T08:08:56.116 INFO:teuthology.orchestra.run.trial066.stdout:(5/37): libcephfs2-20.3.0-5338.g6aeb6975.el9.x8 728 kB/s | 860 kB 00:01 2026-02-25T08:08:56.578 INFO:teuthology.orchestra.run.trial066.stdout:(6/37): libradosstriper1-20.3.0-5338.g6aeb6975. 570 kB/s | 263 kB 00:00 2026-02-25T08:09:00.254 INFO:teuthology.orchestra.run.trial042.stdout:(1/37): ceph-base-20.3.0-5338.g6aeb6975.el9.x86 529 kB/s | 5.1 MB 00:09 2026-02-25T08:09:00.324 INFO:teuthology.orchestra.run.trial042.stdout:(2/37): ceph-selinux-20.3.0-5338.g6aeb6975.el9. 362 kB/s | 25 kB 00:00 2026-02-25T08:09:00.465 INFO:teuthology.orchestra.run.trial042.stdout:(3/37): libcephfs-daemon-20.3.0-5338.g6aeb6975. 262 kB/s | 37 kB 00:00 2026-02-25T08:09:00.536 INFO:teuthology.orchestra.run.trial042.stdout:(4/37): libcephfs-proxy2-20.3.0-5338.g6aeb6975. 360 kB/s | 26 kB 00:00 2026-02-25T08:09:03.033 INFO:teuthology.orchestra.run.trial042.stdout:(5/37): libcephfs2-20.3.0-5338.g6aeb6975.el9.x8 345 kB/s | 860 kB 00:02 2026-02-25T08:09:03.897 INFO:teuthology.orchestra.run.trial042.stdout:(6/37): libradosstriper1-20.3.0-5338.g6aeb6975. 304 kB/s | 263 kB 00:00 2026-02-25T08:09:05.181 INFO:teuthology.orchestra.run.trial042.stdout:(7/37): ceph-radosgw-20.3.0-5338.g6aeb6975.el9. 773 kB/s | 11 MB 00:14 2026-02-25T08:09:05.534 INFO:teuthology.orchestra.run.trial042.stdout:(8/37): python3-ceph-argparse-20.3.0-5338.g6aeb 127 kB/s | 45 kB 00:00 2026-02-25T08:09:05.899 INFO:teuthology.orchestra.run.trial066.stdout:(7/37): librgw2-20.3.0-5338.g6aeb6975.el9.x86_6 618 kB/s | 5.6 MB 00:09 2026-02-25T08:09:06.068 INFO:teuthology.orchestra.run.trial042.stdout:(9/37): python3-ceph-common-20.3.0-5338.g6aeb69 347 kB/s | 185 kB 00:00 2026-02-25T08:09:06.110 INFO:teuthology.orchestra.run.trial066.stdout:(8/37): python3-ceph-argparse-20.3.0-5338.g6aeb 213 kB/s | 45 kB 00:00 2026-02-25T08:09:06.814 INFO:teuthology.orchestra.run.trial066.stdout:(9/37): python3-ceph-common-20.3.0-5338.g6aeb69 263 kB/s | 185 kB 00:00 2026-02-25T08:09:06.899 INFO:teuthology.orchestra.run.trial042.stdout:(10/37): python3-cephfs-20.3.0-5338.g6aeb6975.e 245 kB/s | 204 kB 00:00 2026-02-25T08:09:07.497 INFO:teuthology.orchestra.run.trial066.stdout:(10/37): python3-cephfs-20.3.0-5338.g6aeb6975.e 298 kB/s | 204 kB 00:00 2026-02-25T08:09:07.870 INFO:teuthology.orchestra.run.trial042.stdout:(11/37): python3-rados-20.3.0-5338.g6aeb6975.el 334 kB/s | 324 kB 00:00 2026-02-25T08:09:08.332 INFO:teuthology.orchestra.run.trial066.stdout:(11/37): python3-rados-20.3.0-5338.g6aeb6975.el 388 kB/s | 324 kB 00:00 2026-02-25T08:09:08.558 INFO:teuthology.orchestra.run.trial042.stdout:(12/37): python3-rbd-20.3.0-5338.g6aeb6975.el9. 444 kB/s | 306 kB 00:00 2026-02-25T08:09:08.768 INFO:teuthology.orchestra.run.trial042.stdout:(13/37): python3-rgw-20.3.0-5338.g6aeb6975.el9. 475 kB/s | 100 kB 00:00 2026-02-25T08:09:08.800 INFO:teuthology.orchestra.run.trial042.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 1.0 MB/s | 33 kB 00:00 2026-02-25T08:09:08.845 INFO:teuthology.orchestra.run.trial042.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 26 MB/s | 1.2 MB 00:00 2026-02-25T08:09:08.855 INFO:teuthology.orchestra.run.trial042.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 9.3 MB/s | 93 kB 00:00 2026-02-25T08:09:08.910 INFO:teuthology.orchestra.run.trial066.stdout:(12/37): ceph-radosgw-20.3.0-5338.g6aeb6975.el9 643 kB/s | 11 MB 00:17 2026-02-25T08:09:09.172 INFO:teuthology.orchestra.run.trial066.stdout:(13/37): python3-rbd-20.3.0-5338.g6aeb6975.el9. 364 kB/s | 306 kB 00:00 2026-02-25T08:09:09.190 INFO:teuthology.orchestra.run.trial066.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 1.8 MB/s | 33 kB 00:00 2026-02-25T08:09:09.222 INFO:teuthology.orchestra.run.trial066.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 36 MB/s | 1.2 MB 00:00 2026-02-25T08:09:09.227 INFO:teuthology.orchestra.run.trial042.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:09.229 INFO:teuthology.orchestra.run.trial066.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 14 MB/s | 93 kB 00:00 2026-02-25T08:09:09.468 INFO:teuthology.orchestra.run.trial066.stdout:(17/37): python3-rgw-20.3.0-5338.g6aeb6975.el9. 179 kB/s | 100 kB 00:00 2026-02-25T08:09:09.663 INFO:teuthology.orchestra.run.trial042.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:09.727 INFO:teuthology.orchestra.run.trial066.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:09.761 INFO:teuthology.orchestra.run.trial042.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 115 kB/s | 104 kB 00:00 2026-02-25T08:09:09.883 INFO:teuthology.orchestra.run.trial066.stdout:[MIRROR] libnbd-1.20.3-4.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libnbd-1.20.3-4.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:10.117 INFO:teuthology.orchestra.run.trial042.stdout:[MIRROR] libnbd-1.20.3-4.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libnbd-1.20.3-4.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:10.393 INFO:teuthology.orchestra.run.trial066.stdout:[MIRROR] boost-program-options-1.75.0-13.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-13.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:10.441 INFO:teuthology.orchestra.run.trial066.stdout:(18/37): boost-program-options-1.75.0-13.el9.x8 86 kB/s | 104 kB 00:01 2026-02-25T08:09:10.472 INFO:teuthology.orchestra.run.trial042.stdout:[MIRROR] libnbd-1.20.3-4.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libnbd-1.20.3-4.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:10.519 INFO:teuthology.orchestra.run.trial042.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 216 kB/s | 164 kB 00:00 2026-02-25T08:09:10.550 INFO:teuthology.orchestra.run.trial066.stdout:[MIRROR] libnbd-1.20.3-4.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libnbd-1.20.3-4.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:10.575 INFO:teuthology.orchestra.run.trial066.stdout:(19/37): libnbd-1.20.3-4.el9.x86_64.rpm 148 kB/s | 164 kB 00:01 2026-02-25T08:09:10.856 INFO:teuthology.orchestra.run.trial066.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:10.873 INFO:teuthology.orchestra.run.trial042.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:10.990 INFO:teuthology.orchestra.run.trial066.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:11.226 INFO:teuthology.orchestra.run.trial042.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:11.254 INFO:teuthology.orchestra.run.trial042.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 218 kB/s | 160 kB 00:00 2026-02-25T08:09:11.275 INFO:teuthology.orchestra.run.trial066.stdout:[MIRROR] libpmemobj-1.12.1-1.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/libpmemobj-1.12.1-1.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:11.303 INFO:teuthology.orchestra.run.trial066.stdout:(20/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 186 kB/s | 160 kB 00:00 2026-02-25T08:09:11.329 INFO:teuthology.orchestra.run.trial066.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 26 MB/s | 662 kB 00:00 2026-02-25T08:09:11.351 INFO:teuthology.orchestra.run.trial066.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 13 MB/s | 292 kB 00:00 2026-02-25T08:09:11.373 INFO:teuthology.orchestra.run.trial066.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 11 MB/s | 240 kB 00:00 2026-02-25T08:09:11.395 INFO:teuthology.orchestra.run.trial066.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.6 MB/s | 37 kB 00:00 2026-02-25T08:09:11.414 INFO:teuthology.orchestra.run.trial066.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:11.418 INFO:teuthology.orchestra.run.trial066.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 3.0 MB/s | 66 kB 00:00 2026-02-25T08:09:11.435 INFO:teuthology.orchestra.run.trial066.stdout:(26/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 53 kB/s | 45 kB 00:00 2026-02-25T08:09:11.608 INFO:teuthology.orchestra.run.trial042.stdout:[MIRROR] librabbitmq-0.11.0-7.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/librabbitmq-0.11.0-7.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:11.631 INFO:teuthology.orchestra.run.trial042.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 120 kB/s | 45 kB 00:00 2026-02-25T08:09:11.690 INFO:teuthology.orchestra.run.trial042.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2026-02-25T08:09:11.716 INFO:teuthology.orchestra.run.trial042.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 11 MB/s | 292 kB 00:00 2026-02-25T08:09:11.739 INFO:teuthology.orchestra.run.trial042.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 9.9 MB/s | 240 kB 00:00 2026-02-25T08:09:11.764 INFO:teuthology.orchestra.run.trial042.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.5 MB/s | 37 kB 00:00 2026-02-25T08:09:11.784 INFO:teuthology.orchestra.run.trial042.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 3.2 MB/s | 66 kB 00:00 2026-02-25T08:09:11.881 INFO:teuthology.orchestra.run.trial066.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 665 kB/s | 308 kB 00:00 2026-02-25T08:09:11.950 INFO:teuthology.orchestra.run.trial066.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 356 kB/s | 25 kB 00:00 2026-02-25T08:09:12.021 INFO:teuthology.orchestra.run.trial066.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 692 kB/s | 49 kB 00:00 2026-02-25T08:09:12.090 INFO:teuthology.orchestra.run.trial066.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 971 kB/s | 67 kB 00:00 2026-02-25T08:09:12.279 INFO:teuthology.orchestra.run.trial066.stdout:(31/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2026-02-25T08:09:12.295 INFO:teuthology.orchestra.run.trial042.stdout:(26/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 602 kB/s | 308 kB 00:00 2026-02-25T08:09:12.350 INFO:teuthology.orchestra.run.trial066.stdout:(32/37): libarrow-9.0.0-15.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2026-02-25T08:09:12.350 INFO:teuthology.orchestra.run.trial066.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 2.6 MB/s | 191 kB 00:00 2026-02-25T08:09:12.457 INFO:teuthology.orchestra.run.trial066.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2026-02-25T08:09:12.594 INFO:teuthology.orchestra.run.trial042.stdout:(27/37): libarrow-9.0.0-15.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2026-02-25T08:09:12.666 INFO:teuthology.orchestra.run.trial042.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 346 kB/s | 25 kB 00:00 2026-02-25T08:09:12.738 INFO:teuthology.orchestra.run.trial042.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 680 kB/s | 49 kB 00:00 2026-02-25T08:09:12.813 INFO:teuthology.orchestra.run.trial042.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 897 kB/s | 67 kB 00:00 2026-02-25T08:09:12.902 INFO:teuthology.orchestra.run.trial042.stdout:(31/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 9.2 MB/s | 838 kB 00:00 2026-02-25T08:09:12.977 INFO:teuthology.orchestra.run.trial042.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2026-02-25T08:09:13.079 INFO:teuthology.orchestra.run.trial042.stdout:(33/37): thrift-0.15.0-4.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2026-02-25T08:09:20.108 INFO:teuthology.orchestra.run.trial042.stdout:(34/37): librgw2-20.3.0-5338.g6aeb6975.el9.x86_ 356 kB/s | 5.6 MB 00:16 2026-02-25T08:09:20.166 INFO:teuthology.orchestra.run.trial066.stdout:(35/37): librbd1-20.3.0-5338.g6aeb6975.el9.x86_ 357 kB/s | 2.7 MB 00:07 2026-02-25T08:09:21.876 INFO:teuthology.orchestra.run.trial066.stdout:(36/37): librados2-20.3.0-5338.g6aeb6975.el9.x8 353 kB/s | 3.3 MB 00:09 2026-02-25T08:09:22.279 INFO:teuthology.orchestra.run.trial042.stdout:(35/37): librados2-20.3.0-5338.g6aeb6975.el9.x8 365 kB/s | 3.3 MB 00:09 2026-02-25T08:09:28.810 INFO:teuthology.orchestra.run.trial042.stdout:(36/37): librbd1-20.3.0-5338.g6aeb6975.el9.x86_ 316 kB/s | 2.7 MB 00:08 2026-02-25T08:09:43.841 INFO:teuthology.orchestra.run.trial066.stdout:(37/37): ceph-common-20.3.0-5338.g6aeb6975.el9. 404 kB/s | 21 MB 00:52 2026-02-25T08:09:43.846 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:09:43.846 INFO:teuthology.orchestra.run.trial066.stdout:Total 1.2 MB/s | 61 MB 00:53 2026-02-25T08:09:43.937 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:09:43.956 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:09:43.956 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:09:44.204 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:09:44.204 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:09:44.496 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:09:44.573 INFO:teuthology.orchestra.run.trial042.stdout:(37/37): ceph-common-20.3.0-5338.g6aeb6975.el9. 393 kB/s | 21 MB 00:54 2026-02-25T08:09:44.575 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:09:44.576 INFO:teuthology.orchestra.run.trial042.stdout:Total 1.1 MB/s | 61 MB 00:54 2026-02-25T08:09:44.640 INFO:teuthology.orchestra.run.trial066.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-25T08:09:44.644 INFO:teuthology.orchestra.run.trial066.stdout: Upgrading : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:44.666 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:09:44.684 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:09:44.685 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:09:44.694 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:44.695 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:44.707 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:44.711 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 4/39 2026-02-25T08:09:44.711 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:44.740 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:44.744 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 6/39 2026-02-25T08:09:44.752 INFO:teuthology.orchestra.run.trial066.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-25T08:09:44.754 INFO:teuthology.orchestra.run.trial066.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-25T08:09:44.757 INFO:teuthology.orchestra.run.trial066.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-25T08:09:44.761 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-25T08:09:44.773 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 11/39 2026-02-25T08:09:44.785 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:09:44.786 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:09:44.810 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:09:44.814 INFO:teuthology.orchestra.run.trial066.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-25T08:09:44.845 INFO:teuthology.orchestra.run.trial066.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T08:09:44.851 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T08:09:44.857 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T08:09:44.859 INFO:teuthology.orchestra.run.trial066.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T08:09:44.863 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T08:09:44.930 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:09:44.930 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:09:44.980 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:09:44.981 INFO:teuthology.orchestra.run.trial066.stdout: Upgrading : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:09:45.006 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:09:45.009 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 22/39 2026-02-25T08:09:45.015 INFO:teuthology.orchestra.run.trial066.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T08:09:45.229 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:09:45.266 INFO:teuthology.orchestra.run.trial066.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T08:09:45.268 INFO:teuthology.orchestra.run.trial066.stdout: Installing : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:09:45.285 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:09:45.290 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 26/39 2026-02-25T08:09:45.295 INFO:teuthology.orchestra.run.trial066.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T08:09:45.297 INFO:teuthology.orchestra.run.trial066.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T08:09:45.298 INFO:teuthology.orchestra.run.trial066.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:09:45.316 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:09:45.370 INFO:teuthology.orchestra.run.trial042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-25T08:09:45.374 INFO:teuthology.orchestra.run.trial042.stdout: Upgrading : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:45.423 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:45.424 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:45.431 INFO:teuthology.orchestra.run.trial066.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T08:09:45.433 INFO:teuthology.orchestra.run.trial066.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:09:45.436 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:45.438 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 4/39 2026-02-25T08:09:45.439 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:45.466 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:45.469 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 6/39 2026-02-25T08:09:45.476 INFO:teuthology.orchestra.run.trial042.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-25T08:09:45.479 INFO:teuthology.orchestra.run.trial042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-25T08:09:45.482 INFO:teuthology.orchestra.run.trial042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-25T08:09:45.486 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-25T08:09:45.498 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 11/39 2026-02-25T08:09:45.511 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:09:45.511 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:09:45.535 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:09:45.539 INFO:teuthology.orchestra.run.trial042.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-25T08:09:45.572 INFO:teuthology.orchestra.run.trial042.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T08:09:45.575 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T08:09:45.581 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T08:09:45.584 INFO:teuthology.orchestra.run.trial042.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T08:09:45.588 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T08:09:45.703 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:09:45.704 INFO:teuthology.orchestra.run.trial042.stdout: Upgrading : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:09:45.709 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:09:45.709 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T08:09:45.709 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:45.732 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:09:45.735 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 22/39 2026-02-25T08:09:45.741 INFO:teuthology.orchestra.run.trial042.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T08:09:45.754 INFO:teuthology.orchestra.run.trial066.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T08:09:45.756 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 33/39 2026-02-25T08:09:45.989 INFO:teuthology.orchestra.run.trial042.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T08:09:45.990 INFO:teuthology.orchestra.run.trial042.stdout: Installing : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:09:46.006 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:09:46.010 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 26/39 2026-02-25T08:09:46.014 INFO:teuthology.orchestra.run.trial042.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T08:09:46.016 INFO:teuthology.orchestra.run.trial042.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T08:09:46.017 INFO:teuthology.orchestra.run.trial042.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:09:46.033 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:09:46.147 INFO:teuthology.orchestra.run.trial042.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T08:09:46.149 INFO:teuthology.orchestra.run.trial042.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:09:46.439 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:09:46.440 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T08:09:46.440 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:46.484 INFO:teuthology.orchestra.run.trial042.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T08:09:46.486 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 33/39 2026-02-25T08:09:46.713 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:46.716 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:46.981 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:46.984 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:09:47.021 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:09:47.021 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T08:09:47.021 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T08:09:47.022 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:47.025 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:47.430 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:47.432 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:47.693 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:47.696 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:09:47.736 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:09:47.737 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T08:09:47.737 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T08:09:47.737 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:47.740 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:51.456 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:51.457 INFO:teuthology.orchestra.run.trial066.stdout:skipping the directory /sys 2026-02-25T08:09:51.457 INFO:teuthology.orchestra.run.trial066.stdout:skipping the directory /proc 2026-02-25T08:09:51.457 INFO:teuthology.orchestra.run.trial066.stdout:skipping the directory /mnt 2026-02-25T08:09:51.457 INFO:teuthology.orchestra.run.trial066.stdout:skipping the directory /var/tmp 2026-02-25T08:09:51.457 INFO:teuthology.orchestra.run.trial066.stdout:skipping the directory /home 2026-02-25T08:09:51.457 INFO:teuthology.orchestra.run.trial066.stdout:skipping the directory /root 2026-02-25T08:09:51.457 INFO:teuthology.orchestra.run.trial066.stdout:skipping the directory /tmp 2026-02-25T08:09:51.457 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:51.955 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:09:51.979 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:09:51.979 INFO:teuthology.orchestra.run.trial066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:09:51.979 INFO:teuthology.orchestra.run.trial066.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T08:09:51.979 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:09:51.979 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:09:51.979 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:51.981 INFO:teuthology.orchestra.run.trial066.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:09:51.993 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:09:51.993 INFO:teuthology.orchestra.run.trial066.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:52.047 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:52.048 INFO:teuthology.orchestra.run.trial066.stdout:Creating group 'qat' with GID 992. 2026-02-25T08:09:52.048 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:52.261 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:52.261 INFO:teuthology.orchestra.run.trial042.stdout:skipping the directory /sys 2026-02-25T08:09:52.261 INFO:teuthology.orchestra.run.trial042.stdout:skipping the directory /proc 2026-02-25T08:09:52.261 INFO:teuthology.orchestra.run.trial042.stdout:skipping the directory /mnt 2026-02-25T08:09:52.261 INFO:teuthology.orchestra.run.trial042.stdout:skipping the directory /var/tmp 2026-02-25T08:09:52.262 INFO:teuthology.orchestra.run.trial042.stdout:skipping the directory /home 2026-02-25T08:09:52.262 INFO:teuthology.orchestra.run.trial042.stdout:skipping the directory /root 2026-02-25T08:09:52.262 INFO:teuthology.orchestra.run.trial042.stdout:skipping the directory /tmp 2026-02-25T08:09:52.262 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/39 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/39 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/39 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 7/39 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 8/39 2026-02-25T08:09:52.343 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 9/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 10/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 11/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 13/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 14/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 15/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T08:09:52.344 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T08:09:52.345 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T08:09:52.346 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:52.346 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T08:09:52.346 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 38/39 2026-02-25T08:09:52.411 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:52.411 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:52.411 INFO:teuthology.orchestra.run.trial066.stdout:Upgraded: 2026-02-25T08:09:52.411 INFO:teuthology.orchestra.run.trial066.stdout: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.411 INFO:teuthology.orchestra.run.trial066.stdout: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.412 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T08:09:52.413 INFO:teuthology.orchestra.run.trial066.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:52.414 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:09:52.452 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-test 2026-02-25T08:09:52.752 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:09:52.775 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:01:03 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:09:52.776 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:09:52.776 INFO:teuthology.orchestra.run.trial042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:09:52.776 INFO:teuthology.orchestra.run.trial042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T08:09:52.776 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:09:52.776 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:09:52.777 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:52.778 INFO:teuthology.orchestra.run.trial042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:09:52.789 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:09:52.789 INFO:teuthology.orchestra.run.trial042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:52.842 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:52.842 INFO:teuthology.orchestra.run.trial042.stdout:Creating group 'qat' with GID 992. 2026-02-25T08:09:52.842 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:52.857 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:09:52.858 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:09:52.858 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repository Size 2026-02-25T08:09:52.858 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:09:52.858 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:09:52.858 INFO:teuthology.orchestra.run.trial066.stdout: ceph-test x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 127 M 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout:Installing dependencies: 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: libcephsqlite x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 153 k 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T08:09:52.859 INFO:teuthology.orchestra.run.trial066.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T08:09:52.860 INFO:teuthology.orchestra.run.trial066.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T08:09:52.860 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:52.860 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:09:52.860 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:09:52.860 INFO:teuthology.orchestra.run.trial066.stdout:Install 12 Packages 2026-02-25T08:09:52.860 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:09:52.861 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 132 M 2026-02-25T08:09:52.861 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 988 M 2026-02-25T08:09:52.861 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:09:53.129 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/39 2026-02-25T08:09:53.129 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:53.129 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:53.129 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/39 2026-02-25T08:09:53.129 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 7/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 8/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 9/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 10/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 11/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 13/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 14/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 15/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T08:09:53.130 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T08:09:53.131 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T08:09:53.132 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 38/39 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout:Upgraded: 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.197 INFO:teuthology.orchestra.run.trial042.stdout: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T08:09:53.198 INFO:teuthology.orchestra.run.trial042.stdout: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.199 INFO:teuthology.orchestra.run.trial042.stdout: python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout: python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout: python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:53.200 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:09:53.237 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-test 2026-02-25T08:09:53.556 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:01:05 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repository Size 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout: ceph-test x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 127 M 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout:Installing dependencies: 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout: libcephsqlite x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 153 k 2026-02-25T08:09:53.640 INFO:teuthology.orchestra.run.trial042.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout:Install 12 Packages 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 132 M 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 988 M 2026-02-25T08:09:53.641 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:09:53.951 INFO:teuthology.orchestra.run.trial066.stdout:(1/12): libcephsqlite-20.3.0-5338.g6aeb6975.el9 268 kB/s | 153 kB 00:00 2026-02-25T08:09:54.123 INFO:teuthology.orchestra.run.trial066.stdout:(2/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.8 MB/s | 2.1 MB 00:00 2026-02-25T08:09:54.165 INFO:teuthology.orchestra.run.trial042.stdout:(1/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 64 MB/s | 2.1 MB 00:00 2026-02-25T08:09:54.169 INFO:teuthology.orchestra.run.trial042.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 5.4 MB/s | 23 kB 00:00 2026-02-25T08:09:54.174 INFO:teuthology.orchestra.run.trial042.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 18 MB/s | 94 kB 00:00 2026-02-25T08:09:54.196 INFO:teuthology.orchestra.run.trial066.stdout:(3/12): libatomic-11.5.0-14.el9.x86_64.rpm 93 kB/s | 23 kB 00:00 2026-02-25T08:09:54.203 INFO:teuthology.orchestra.run.trial066.stdout:(4/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2026-02-25T08:09:54.326 INFO:teuthology.orchestra.run.trial042.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 3.4 MB/s | 525 kB 00:00 2026-02-25T08:09:54.347 INFO:teuthology.orchestra.run.trial066.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 1.2 MB/s | 176 kB 00:00 2026-02-25T08:09:54.349 INFO:teuthology.orchestra.run.trial042.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 7.5 MB/s | 176 kB 00:00 2026-02-25T08:09:54.373 INFO:teuthology.orchestra.run.trial042.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 9.3 MB/s | 233 kB 00:00 2026-02-25T08:09:54.377 INFO:teuthology.orchestra.run.trial066.stdout:(6/12): libasan8-15.2.1-7.el9.x86_64.rpm 2.8 MB/s | 525 kB 00:00 2026-02-25T08:09:54.399 INFO:teuthology.orchestra.run.trial066.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 4.3 MB/s | 233 kB 00:00 2026-02-25T08:09:54.420 INFO:teuthology.orchestra.run.trial042.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 22 MB/s | 1.0 MB 00:00 2026-02-25T08:09:54.437 INFO:teuthology.orchestra.run.trial066.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 7.8 MB/s | 303 kB 00:00 2026-02-25T08:09:54.442 INFO:teuthology.orchestra.run.trial042.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 14 MB/s | 303 kB 00:00 2026-02-25T08:09:54.442 INFO:teuthology.orchestra.run.trial066.stdout:(9/12): protobuf-3.14.0-17.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2026-02-25T08:09:54.462 INFO:teuthology.orchestra.run.trial042.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 3.1 MB/s | 64 kB 00:00 2026-02-25T08:09:54.464 INFO:teuthology.orchestra.run.trial066.stdout:(10/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.4 MB/s | 64 kB 00:00 2026-02-25T08:09:54.652 INFO:teuthology.orchestra.run.trial042.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 644 kB/s | 122 kB 00:00 2026-02-25T08:09:54.817 INFO:teuthology.orchestra.run.trial066.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 325 kB/s | 122 kB 00:00 2026-02-25T08:09:55.047 INFO:teuthology.orchestra.run.trial042.stdout:(11/12): libcephsqlite-20.3.0-5338.g6aeb6975.el 167 kB/s | 153 kB 00:00 2026-02-25T08:14:59.670 INFO:teuthology.orchestra.run.trial066.stdout:(12/12): ceph-test-20.3.0-5338.g6aeb6975.el9.x8 424 kB/s | 127 MB 05:06 2026-02-25T08:14:59.676 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:14:59.676 INFO:teuthology.orchestra.run.trial066.stdout:Total 439 kB/s | 132 MB 05:06 2026-02-25T08:14:59.738 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:14:59.750 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:14:59.750 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:15:00.182 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:15:00.182 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:15:00.642 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:15:00.650 INFO:teuthology.orchestra.run.trial066.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-25T08:15:00.659 INFO:teuthology.orchestra.run.trial066.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-25T08:15:00.664 INFO:teuthology.orchestra.run.trial066.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-25T08:15:00.670 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-25T08:15:00.673 INFO:teuthology.orchestra.run.trial066.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-25T08:15:00.677 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-25T08:15:00.678 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:15:00.694 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:15:00.696 INFO:teuthology.orchestra.run.trial066.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-25T08:15:00.706 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-25T08:15:00.714 INFO:teuthology.orchestra.run.trial066.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-25T08:15:00.715 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:15:07.662 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:15:07.680 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:15:08.441 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:15:08.441 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-25T08:15:08.442 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-25T08:15:08.502 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-25T08:15:08.502 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:15:08.502 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T08:15:08.503 INFO:teuthology.orchestra.run.trial066.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T08:15:08.504 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:15:08.504 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:15:08.545 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph 2026-02-25T08:15:08.864 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:06:19 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:15:08.951 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:15:08.954 INFO:teuthology.orchestra.run.trial066.stdout:======================================================================================= 2026-02-25T08:15:08.954 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repository Size 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout:======================================================================================= 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout: ceph x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 6.5 k 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout:Installing dependencies: 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 57 k 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mds x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.0 M 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 937 k 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 297 k 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 4.9 M 2026-02-25T08:15:08.955 INFO:teuthology.orchestra.run.trial066.stdout: ceph-osd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 163 k 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: ceph-osd-classic x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 15 M 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: ceph-osd-crimson x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 88 M 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 17 k 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 1.0 M 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T08:15:08.956 INFO:teuthology.orchestra.run.trial066.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T08:15:08.957 INFO:teuthology.orchestra.run.trial066.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T08:15:08.958 INFO:teuthology.orchestra.run.trial066.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T08:15:08.959 INFO:teuthology.orchestra.run.trial066.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T08:15:08.960 INFO:teuthology.orchestra.run.trial066.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T08:15:08.961 INFO:teuthology.orchestra.run.trial066.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout:Installing weak dependencies: 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 184 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 51 M 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 7.4 M 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 22 k 2026-02-25T08:15:08.962 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-rook noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 50 k 2026-02-25T08:15:08.963 INFO:teuthology.orchestra.run.trial066.stdout: ceph-volume noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 301 k 2026-02-25T08:15:08.963 INFO:teuthology.orchestra.run.trial066.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T08:15:08.963 INFO:teuthology.orchestra.run.trial066.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T08:15:08.963 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:15:08.963 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:15:08.963 INFO:teuthology.orchestra.run.trial066.stdout:======================================================================================= 2026-02-25T08:15:08.963 INFO:teuthology.orchestra.run.trial066.stdout:Install 96 Packages 2026-02-25T08:15:08.963 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:15:08.964 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 226 M 2026-02-25T08:15:08.964 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 1.5 G 2026-02-25T08:15:08.964 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:15:09.970 INFO:teuthology.orchestra.run.trial066.stdout:(1/96): ceph-20.3.0-5338.g6aeb6975.el9.x86_64.r 23 kB/s | 6.5 kB 00:00 2026-02-25T08:15:15.983 INFO:teuthology.orchestra.run.trial066.stdout:(2/96): ceph-mgr-20.3.0-5338.g6aeb6975.el9.x86_ 149 kB/s | 937 kB 00:06 2026-02-25T08:15:17.422 INFO:teuthology.orchestra.run.trial066.stdout:(3/96): ceph-osd-20.3.0-5338.g6aeb6975.el9.x86_ 114 kB/s | 163 kB 00:01 2026-02-25T08:15:18.183 INFO:teuthology.orchestra.run.trial066.stdout:(4/96): ceph-mds-20.3.0-5338.g6aeb6975.el9.x86_ 244 kB/s | 2.0 MB 00:08 2026-02-25T08:15:18.365 INFO:teuthology.orchestra.run.trial042.stdout:(12/12): ceph-test-20.3.0-5338.g6aeb6975.el9.x8 401 kB/s | 127 MB 05:24 2026-02-25T08:15:18.367 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:15:18.367 INFO:teuthology.orchestra.run.trial042.stdout:Total 415 kB/s | 132 MB 05:24 2026-02-25T08:15:18.435 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:15:18.446 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:15:18.446 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:15:18.879 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:15:18.879 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:15:19.338 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:15:19.348 INFO:teuthology.orchestra.run.trial042.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-25T08:15:19.356 INFO:teuthology.orchestra.run.trial042.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-25T08:15:19.362 INFO:teuthology.orchestra.run.trial042.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-25T08:15:19.366 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-25T08:15:19.369 INFO:teuthology.orchestra.run.trial042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-25T08:15:19.373 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-25T08:15:19.374 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:15:19.391 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:15:19.393 INFO:teuthology.orchestra.run.trial042.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-25T08:15:19.404 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-25T08:15:19.413 INFO:teuthology.orchestra.run.trial042.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-25T08:15:19.413 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:15:19.635 INFO:teuthology.orchestra.run.trial066.stdout:(5/96): ceph-mon-20.3.0-5338.g6aeb6975.el9.x86_ 516 kB/s | 4.9 MB 00:09 2026-02-25T08:15:19.886 INFO:teuthology.orchestra.run.trial066.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5338.g6a 228 kB/s | 57 kB 00:00 2026-02-25T08:15:20.655 INFO:teuthology.orchestra.run.trial066.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5338.g6aeb6975. 239 kB/s | 184 kB 00:00 2026-02-25T08:15:26.357 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:15:26.375 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-25T08:15:27.149 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-25T08:15:27.150 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-25T08:15:27.150 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-25T08:15:27.150 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-25T08:15:27.208 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-25T08:15:27.208 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:15:27.208 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:15:27.208 INFO:teuthology.orchestra.run.trial042.stdout: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:27.208 INFO:teuthology.orchestra.run.trial042.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T08:15:27.208 INFO:teuthology.orchestra.run.trial042.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:15:27.209 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:15:27.251 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph 2026-02-25T08:15:27.565 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:06:39 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:15:27.653 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:15:27.656 INFO:teuthology.orchestra.run.trial042.stdout:======================================================================================= 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repository Size 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout:======================================================================================= 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: ceph x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 6.5 k 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout:Installing dependencies: 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 57 k 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mds x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.0 M 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 937 k 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 297 k 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 4.9 M 2026-02-25T08:15:27.657 INFO:teuthology.orchestra.run.trial042.stdout: ceph-osd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 163 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: ceph-osd-classic x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 15 M 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: ceph-osd-crimson x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 88 M 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 17 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 1.0 M 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T08:15:27.658 INFO:teuthology.orchestra.run.trial042.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T08:15:27.659 INFO:teuthology.orchestra.run.trial042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T08:15:27.660 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T08:15:27.661 INFO:teuthology.orchestra.run.trial042.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T08:15:27.662 INFO:teuthology.orchestra.run.trial042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T08:15:27.663 INFO:teuthology.orchestra.run.trial042.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout:Installing weak dependencies: 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 184 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 51 M 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 7.4 M 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 22 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-rook noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 50 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: ceph-volume noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 301 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T08:15:27.664 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:15:27.665 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:15:27.665 INFO:teuthology.orchestra.run.trial042.stdout:======================================================================================= 2026-02-25T08:15:27.665 INFO:teuthology.orchestra.run.trial042.stdout:Install 96 Packages 2026-02-25T08:15:27.665 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:15:27.666 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 226 M 2026-02-25T08:15:27.666 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 1.5 G 2026-02-25T08:15:27.666 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:15:28.827 INFO:teuthology.orchestra.run.trial042.stdout:(1/96): ceph-20.3.0-5338.g6aeb6975.el9.x86_64.r 10 kB/s | 6.5 kB 00:00 2026-02-25T08:15:30.148 INFO:teuthology.orchestra.run.trial042.stdout:(2/96): ceph-mds-20.3.0-5338.g6aeb6975.el9.x86_ 1.0 MB/s | 2.0 MB 00:01 2026-02-25T08:15:30.641 INFO:teuthology.orchestra.run.trial042.stdout:(3/96): ceph-osd-20.3.0-5338.g6aeb6975.el9.x86_ 331 kB/s | 163 kB 00:00 2026-02-25T08:15:32.221 INFO:teuthology.orchestra.run.trial042.stdout:(4/96): ceph-mgr-20.3.0-5338.g6aeb6975.el9.x86_ 232 kB/s | 937 kB 00:04 2026-02-25T08:15:39.509 INFO:teuthology.orchestra.run.trial042.stdout:(5/96): ceph-mon-20.3.0-5338.g6aeb6975.el9.x86_ 467 kB/s | 4.9 MB 00:10 2026-02-25T08:15:39.792 INFO:teuthology.orchestra.run.trial042.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5338.g6a 202 kB/s | 57 kB 00:00 2026-02-25T08:15:40.493 INFO:teuthology.orchestra.run.trial042.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5338.g6aeb6975. 262 kB/s | 184 kB 00:00 2026-02-25T08:16:11.740 INFO:teuthology.orchestra.run.trial042.stdout:(8/96): ceph-osd-classic-20.3.0-5338.g6aeb6975. 368 kB/s | 15 MB 00:41 2026-02-25T08:16:17.907 INFO:teuthology.orchestra.run.trial066.stdout:(8/96): ceph-osd-classic-20.3.0-5338.g6aeb6975. 250 kB/s | 15 MB 01:00 2026-02-25T08:16:31.761 INFO:teuthology.orchestra.run.trial042.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 379 kB/s | 7.4 MB 00:20 2026-02-25T08:16:31.833 INFO:teuthology.orchestra.run.trial042.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5338.g6aeb69 311 kB/s | 22 kB 00:00 2026-02-25T08:16:32.644 INFO:teuthology.orchestra.run.trial042.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5338.g6ae 366 kB/s | 297 kB 00:00 2026-02-25T08:16:32.856 INFO:teuthology.orchestra.run.trial042.stdout:(12/96): ceph-mgr-rook-20.3.0-5338.g6aeb6975.el 238 kB/s | 50 kB 00:00 2026-02-25T08:16:32.927 INFO:teuthology.orchestra.run.trial042.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5338.g6a 246 kB/s | 17 kB 00:00 2026-02-25T08:16:33.764 INFO:teuthology.orchestra.run.trial042.stdout:(14/96): ceph-volume-20.3.0-5338.g6aeb6975.el9. 360 kB/s | 301 kB 00:00 2026-02-25T08:16:36.282 INFO:teuthology.orchestra.run.trial042.stdout:(15/96): cephadm-20.3.0-5338.g6aeb6975.el9.noar 413 kB/s | 1.0 MB 00:02 2026-02-25T08:16:36.388 INFO:teuthology.orchestra.run.trial042.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 3.2 MB/s | 351 kB 00:00 2026-02-25T08:16:36.410 INFO:teuthology.orchestra.run.trial042.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.8 MB/s | 40 kB 00:00 2026-02-25T08:16:36.432 INFO:teuthology.orchestra.run.trial042.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 3.2 MB/s | 72 kB 00:00 2026-02-25T08:16:36.476 INFO:teuthology.orchestra.run.trial042.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 18 MB/s | 794 kB 00:00 2026-02-25T08:16:36.501 INFO:teuthology.orchestra.run.trial042.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 7.2 MB/s | 184 kB 00:00 2026-02-25T08:16:36.526 INFO:teuthology.orchestra.run.trial042.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 10 MB/s | 253 kB 00:00 2026-02-25T08:16:36.600 INFO:teuthology.orchestra.run.trial042.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 17 MB/s | 1.2 MB 00:00 2026-02-25T08:16:36.624 INFO:teuthology.orchestra.run.trial042.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 4.4 MB/s | 106 kB 00:00 2026-02-25T08:16:36.649 INFO:teuthology.orchestra.run.trial042.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 5.4 MB/s | 135 kB 00:00 2026-02-25T08:16:36.673 INFO:teuthology.orchestra.run.trial042.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 6.0 MB/s | 150 kB 00:00 2026-02-25T08:16:36.698 INFO:teuthology.orchestra.run.trial042.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 5.0 MB/s | 126 kB 00:00 2026-02-25T08:16:36.722 INFO:teuthology.orchestra.run.trial042.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 8.7 MB/s | 218 kB 00:00 2026-02-25T08:16:36.747 INFO:teuthology.orchestra.run.trial042.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 7.1 MB/s | 182 kB 00:00 2026-02-25T08:16:36.771 INFO:teuthology.orchestra.run.trial042.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 11 MB/s | 266 kB 00:00 2026-02-25T08:16:36.794 INFO:teuthology.orchestra.run.trial042.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 1.3 MB/s | 30 kB 00:00 2026-02-25T08:16:36.949 INFO:teuthology.orchestra.run.trial042.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 19 MB/s | 3.0 MB 00:00 2026-02-25T08:16:36.970 INFO:teuthology.orchestra.run.trial042.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 683 kB/s | 15 kB 00:00 2026-02-25T08:16:36.994 INFO:teuthology.orchestra.run.trial042.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 10 MB/s | 246 kB 00:00 2026-02-25T08:16:37.019 INFO:teuthology.orchestra.run.trial042.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 7.4 MB/s | 188 kB 00:00 2026-02-25T08:16:37.041 INFO:teuthology.orchestra.run.trial042.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-25T08:16:37.236 INFO:teuthology.orchestra.run.trial042.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 27 MB/s | 5.3 MB 00:00 2026-02-25T08:16:37.452 INFO:teuthology.orchestra.run.trial042.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 6.0 MB 00:00 2026-02-25T08:16:37.477 INFO:teuthology.orchestra.run.trial042.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 9.6 MB/s | 244 kB 00:00 2026-02-25T08:16:37.501 INFO:teuthology.orchestra.run.trial042.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 10 MB/s | 249 kB 00:00 2026-02-25T08:16:37.524 INFO:teuthology.orchestra.run.trial042.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 2.1 MB/s | 48 kB 00:00 2026-02-25T08:16:37.547 INFO:teuthology.orchestra.run.trial042.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 7.3 MB/s | 177 kB 00:00 2026-02-25T08:16:37.595 INFO:teuthology.orchestra.run.trial042.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 26 MB/s | 1.2 MB 00:00 2026-02-25T08:16:37.617 INFO:teuthology.orchestra.run.trial042.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 1.5 MB/s | 35 kB 00:00 2026-02-25T08:16:37.832 INFO:teuthology.orchestra.run.trial042.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 29 MB/s | 6.1 MB 00:00 2026-02-25T08:16:37.858 INFO:teuthology.orchestra.run.trial042.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 17 MB/s | 442 kB 00:00 2026-02-25T08:16:37.881 INFO:teuthology.orchestra.run.trial042.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 3.3 MB/s | 77 kB 00:00 2026-02-25T08:16:37.904 INFO:teuthology.orchestra.run.trial042.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 11 MB/s | 267 kB 00:00 2026-02-25T08:16:37.928 INFO:teuthology.orchestra.run.trial042.stdout:(48/96): python3-pyasn1-0.4.8-7.el9.noarch.rpm 6.5 MB/s | 157 kB 00:00 2026-02-25T08:16:37.952 INFO:teuthology.orchestra.run.trial042.stdout:(49/96): python3-pyasn1-modules-0.4.8-7.el9.noa 11 MB/s | 277 kB 00:00 2026-02-25T08:16:37.975 INFO:teuthology.orchestra.run.trial042.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 2.3 MB/s | 54 kB 00:00 2026-02-25T08:16:38.595 INFO:teuthology.orchestra.run.trial042.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 31 MB/s | 19 MB 00:00 2026-02-25T08:16:38.617 INFO:teuthology.orchestra.run.trial042.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-25T08:16:38.640 INFO:teuthology.orchestra.run.trial042.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 8.1 MB/s | 189 kB 00:00 2026-02-25T08:16:38.663 INFO:teuthology.orchestra.run.trial042.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.9 MB/s | 90 kB 00:00 2026-02-25T08:16:38.684 INFO:teuthology.orchestra.run.trial042.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 22 kB 00:00 2026-02-25T08:16:38.703 INFO:teuthology.orchestra.run.trial042.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 44 MB/s | 862 kB 00:00 2026-02-25T08:16:38.739 INFO:teuthology.orchestra.run.trial042.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 15 MB/s | 551 kB 00:00 2026-02-25T08:16:38.742 INFO:teuthology.orchestra.run.trial042.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 5.4 MB/s | 19 kB 00:00 2026-02-25T08:16:38.746 INFO:teuthology.orchestra.run.trial042.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 38 MB/s | 151 kB 00:00 2026-02-25T08:16:38.752 INFO:teuthology.orchestra.run.trial042.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 95 MB/s | 548 kB 00:00 2026-02-25T08:16:38.755 INFO:teuthology.orchestra.run.trial042.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 8.4 MB/s | 29 kB 00:00 2026-02-25T08:16:38.759 INFO:teuthology.orchestra.run.trial042.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 16 MB/s | 60 kB 00:00 2026-02-25T08:16:38.763 INFO:teuthology.orchestra.run.trial042.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 12 MB/s | 43 kB 00:00 2026-02-25T08:16:38.766 INFO:teuthology.orchestra.run.trial042.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 9.2 MB/s | 32 kB 00:00 2026-02-25T08:16:38.770 INFO:teuthology.orchestra.run.trial042.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 4.0 MB/s | 14 kB 00:00 2026-02-25T08:16:38.774 INFO:teuthology.orchestra.run.trial042.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 42 MB/s | 173 kB 00:00 2026-02-25T08:16:38.779 INFO:teuthology.orchestra.run.trial042.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 82 MB/s | 358 kB 00:00 2026-02-25T08:16:38.783 INFO:teuthology.orchestra.run.trial042.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 62 MB/s | 254 kB 00:00 2026-02-25T08:16:38.793 INFO:teuthology.orchestra.run.trial042.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 198 MB/s | 2.0 MB 00:00 2026-02-25T08:16:38.797 INFO:teuthology.orchestra.run.trial042.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 37 MB/s | 144 kB 00:00 2026-02-25T08:16:38.801 INFO:teuthology.orchestra.run.trial042.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 36 MB/s | 139 kB 00:00 2026-02-25T08:16:38.805 INFO:teuthology.orchestra.run.trial042.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 15 MB/s | 56 kB 00:00 2026-02-25T08:16:38.808 INFO:teuthology.orchestra.run.trial042.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 3.0 MB/s | 11 kB 00:00 2026-02-25T08:16:38.812 INFO:teuthology.orchestra.run.trial042.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 5.1 MB/s | 18 kB 00:00 2026-02-25T08:16:38.816 INFO:teuthology.orchestra.run.trial042.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 6.4 MB/s | 23 kB 00:00 2026-02-25T08:16:38.819 INFO:teuthology.orchestra.run.trial042.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 5.4 MB/s | 20 kB 00:00 2026-02-25T08:16:38.823 INFO:teuthology.orchestra.run.trial042.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 5.3 MB/s | 19 kB 00:00 2026-02-25T08:16:38.826 INFO:teuthology.orchestra.run.trial042.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 7.4 MB/s | 26 kB 00:00 2026-02-25T08:16:38.833 INFO:teuthology.orchestra.run.trial042.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 171 MB/s | 1.0 MB 00:00 2026-02-25T08:16:38.836 INFO:teuthology.orchestra.run.trial042.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 21 MB/s | 79 kB 00:00 2026-02-25T08:16:38.840 INFO:teuthology.orchestra.run.trial042.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 22 MB/s | 86 kB 00:00 2026-02-25T08:16:38.844 INFO:teuthology.orchestra.run.trial042.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 16 MB/s | 58 kB 00:00 2026-02-25T08:16:38.847 INFO:teuthology.orchestra.run.trial042.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 4.7 MB/s | 16 kB 00:00 2026-02-25T08:16:38.851 INFO:teuthology.orchestra.run.trial042.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 24 MB/s | 90 kB 00:00 2026-02-25T08:16:38.855 INFO:teuthology.orchestra.run.trial042.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 8.8 MB/s | 31 kB 00:00 2026-02-25T08:16:38.859 INFO:teuthology.orchestra.run.trial042.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 46 MB/s | 188 kB 00:00 2026-02-25T08:16:38.862 INFO:teuthology.orchestra.run.trial042.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 17 MB/s | 59 kB 00:00 2026-02-25T08:16:38.866 INFO:teuthology.orchestra.run.trial042.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 31 MB/s | 125 kB 00:00 2026-02-25T08:16:38.870 INFO:teuthology.orchestra.run.trial042.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 9.6 MB/s | 36 kB 00:00 2026-02-25T08:16:38.874 INFO:teuthology.orchestra.run.trial042.stdout:(90/96): python3-typing-extensions-4.15.0-1.el9 22 MB/s | 86 kB 00:00 2026-02-25T08:16:38.878 INFO:teuthology.orchestra.run.trial042.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 24 MB/s | 90 kB 00:00 2026-02-25T08:16:38.882 INFO:teuthology.orchestra.run.trial042.stdout:(92/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 13 MB/s | 48 kB 00:00 2026-02-25T08:16:38.885 INFO:teuthology.orchestra.run.trial042.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 6.2 MB/s | 22 kB 00:00 2026-02-25T08:16:38.889 INFO:teuthology.orchestra.run.trial042.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 5.6 MB/s | 20 kB 00:00 2026-02-25T08:16:51.996 INFO:teuthology.orchestra.run.trial066.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 223 kB/s | 7.4 MB 00:34 2026-02-25T08:16:52.278 INFO:teuthology.orchestra.run.trial066.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5338.g6aeb69 79 kB/s | 22 kB 00:00 2026-02-25T08:16:53.714 INFO:teuthology.orchestra.run.trial066.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5338.g6ae 207 kB/s | 297 kB 00:01 2026-02-25T08:16:53.994 INFO:teuthology.orchestra.run.trial066.stdout:(12/96): ceph-mgr-rook-20.3.0-5338.g6aeb6975.el 180 kB/s | 50 kB 00:00 2026-02-25T08:16:54.064 INFO:teuthology.orchestra.run.trial066.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5338.g6a 249 kB/s | 17 kB 00:00 2026-02-25T08:16:55.296 INFO:teuthology.orchestra.run.trial066.stdout:(14/96): ceph-volume-20.3.0-5338.g6aeb6975.el9. 245 kB/s | 301 kB 00:01 2026-02-25T08:16:59.478 INFO:teuthology.orchestra.run.trial066.stdout:(15/96): cephadm-20.3.0-5338.g6aeb6975.el9.noar 249 kB/s | 1.0 MB 00:04 2026-02-25T08:17:00.253 INFO:teuthology.orchestra.run.trial066.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 453 kB/s | 351 kB 00:00 2026-02-25T08:17:00.807 INFO:teuthology.orchestra.run.trial066.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 73 kB/s | 40 kB 00:00 2026-02-25T08:17:01.407 INFO:teuthology.orchestra.run.trial066.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 120 kB/s | 72 kB 00:00 2026-02-25T08:17:02.288 INFO:teuthology.orchestra.run.trial066.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 902 kB/s | 794 kB 00:00 2026-02-25T08:17:03.015 INFO:teuthology.orchestra.run.trial066.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 254 kB/s | 184 kB 00:00 2026-02-25T08:17:03.745 INFO:teuthology.orchestra.run.trial066.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 347 kB/s | 253 kB 00:00 2026-02-25T08:17:04.653 INFO:teuthology.orchestra.run.trial066.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 1.4 MB/s | 1.2 MB 00:00 2026-02-25T08:17:05.316 INFO:teuthology.orchestra.run.trial066.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 161 kB/s | 106 kB 00:00 2026-02-25T08:17:06.010 INFO:teuthology.orchestra.run.trial066.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 195 kB/s | 135 kB 00:00 2026-02-25T08:17:06.716 INFO:teuthology.orchestra.run.trial066.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 213 kB/s | 150 kB 00:00 2026-02-25T08:17:07.379 INFO:teuthology.orchestra.run.trial066.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 191 kB/s | 126 kB 00:00 2026-02-25T08:17:08.123 INFO:teuthology.orchestra.run.trial066.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 293 kB/s | 218 kB 00:00 2026-02-25T08:17:08.828 INFO:teuthology.orchestra.run.trial066.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 258 kB/s | 182 kB 00:00 2026-02-25T08:17:09.565 INFO:teuthology.orchestra.run.trial066.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 360 kB/s | 266 kB 00:00 2026-02-25T08:17:09.634 INFO:teuthology.orchestra.run.trial066.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 429 kB/s | 30 kB 00:00 2026-02-25T08:17:09.783 INFO:teuthology.orchestra.run.trial066.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 20 MB/s | 3.0 MB 00:00 2026-02-25T08:17:09.803 INFO:teuthology.orchestra.run.trial066.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 757 kB/s | 15 kB 00:00 2026-02-25T08:17:09.824 INFO:teuthology.orchestra.run.trial066.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 11 MB/s | 246 kB 00:00 2026-02-25T08:17:09.846 INFO:teuthology.orchestra.run.trial066.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 8.6 MB/s | 188 kB 00:00 2026-02-25T08:17:09.866 INFO:teuthology.orchestra.run.trial066.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 2.1 MB/s | 42 kB 00:00 2026-02-25T08:17:09.920 INFO:teuthology.orchestra.run.trial066.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 97 MB/s | 5.3 MB 00:00 2026-02-25T08:17:09.974 INFO:teuthology.orchestra.run.trial066.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 110 MB/s | 6.0 MB 00:00 2026-02-25T08:17:09.997 INFO:teuthology.orchestra.run.trial066.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 11 MB/s | 244 kB 00:00 2026-02-25T08:17:10.020 INFO:teuthology.orchestra.run.trial066.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 11 MB/s | 249 kB 00:00 2026-02-25T08:17:10.040 INFO:teuthology.orchestra.run.trial066.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 2.3 MB/s | 48 kB 00:00 2026-02-25T08:17:10.061 INFO:teuthology.orchestra.run.trial066.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 8.1 MB/s | 177 kB 00:00 2026-02-25T08:17:10.087 INFO:teuthology.orchestra.run.trial066.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 48 MB/s | 1.2 MB 00:00 2026-02-25T08:17:10.108 INFO:teuthology.orchestra.run.trial066.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 1.7 MB/s | 35 kB 00:00 2026-02-25T08:17:10.163 INFO:teuthology.orchestra.run.trial066.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 112 MB/s | 6.1 MB 00:00 2026-02-25T08:17:10.186 INFO:teuthology.orchestra.run.trial066.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 19 MB/s | 442 kB 00:00 2026-02-25T08:17:10.207 INFO:teuthology.orchestra.run.trial066.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 3.6 MB/s | 77 kB 00:00 2026-02-25T08:17:10.228 INFO:teuthology.orchestra.run.trial066.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 12 MB/s | 267 kB 00:00 2026-02-25T08:17:10.250 INFO:teuthology.orchestra.run.trial066.stdout:(48/96): python3-pyasn1-0.4.8-7.el9.noarch.rpm 7.0 MB/s | 157 kB 00:00 2026-02-25T08:17:10.274 INFO:teuthology.orchestra.run.trial066.stdout:(49/96): python3-pyasn1-modules-0.4.8-7.el9.noa 12 MB/s | 277 kB 00:00 2026-02-25T08:17:10.294 INFO:teuthology.orchestra.run.trial066.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 2.5 MB/s | 54 kB 00:00 2026-02-25T08:17:10.454 INFO:teuthology.orchestra.run.trial066.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 121 MB/s | 19 MB 00:00 2026-02-25T08:17:10.475 INFO:teuthology.orchestra.run.trial066.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 2.0 MB/s | 42 kB 00:00 2026-02-25T08:17:10.496 INFO:teuthology.orchestra.run.trial066.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 8.8 MB/s | 189 kB 00:00 2026-02-25T08:17:10.517 INFO:teuthology.orchestra.run.trial066.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 4.3 MB/s | 90 kB 00:00 2026-02-25T08:17:11.228 INFO:teuthology.orchestra.run.trial066.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2026-02-25T08:17:12.453 INFO:teuthology.orchestra.run.trial066.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 703 kB/s | 862 kB 00:01 2026-02-25T08:17:12.815 INFO:teuthology.orchestra.run.trial066.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.5 MB/s | 551 kB 00:00 2026-02-25T08:17:12.858 INFO:teuthology.orchestra.run.trial066.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 459 kB/s | 19 kB 00:00 2026-02-25T08:17:12.907 INFO:teuthology.orchestra.run.trial066.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2026-02-25T08:17:12.978 INFO:teuthology.orchestra.run.trial066.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 7.6 MB/s | 548 kB 00:00 2026-02-25T08:17:13.020 INFO:teuthology.orchestra.run.trial066.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 693 kB/s | 29 kB 00:00 2026-02-25T08:17:13.065 INFO:teuthology.orchestra.run.trial066.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 1.3 MB/s | 60 kB 00:00 2026-02-25T08:17:13.108 INFO:teuthology.orchestra.run.trial066.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2026-02-25T08:17:13.151 INFO:teuthology.orchestra.run.trial066.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 758 kB/s | 32 kB 00:00 2026-02-25T08:17:13.193 INFO:teuthology.orchestra.run.trial066.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 335 kB/s | 14 kB 00:00 2026-02-25T08:17:13.242 INFO:teuthology.orchestra.run.trial066.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 3.4 MB/s | 173 kB 00:00 2026-02-25T08:17:13.300 INFO:teuthology.orchestra.run.trial066.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 6.1 MB/s | 358 kB 00:00 2026-02-25T08:17:13.353 INFO:teuthology.orchestra.run.trial066.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 4.7 MB/s | 254 kB 00:00 2026-02-25T08:17:13.483 INFO:teuthology.orchestra.run.trial066.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 16 MB/s | 2.0 MB 00:00 2026-02-25T08:17:13.526 INFO:teuthology.orchestra.run.trial066.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 3.2 MB/s | 144 kB 00:00 2026-02-25T08:17:13.570 INFO:teuthology.orchestra.run.trial066.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 3.1 MB/s | 139 kB 00:00 2026-02-25T08:17:13.612 INFO:teuthology.orchestra.run.trial066.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 1.3 MB/s | 56 kB 00:00 2026-02-25T08:17:13.654 INFO:teuthology.orchestra.run.trial066.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 255 kB/s | 11 kB 00:00 2026-02-25T08:17:13.697 INFO:teuthology.orchestra.run.trial066.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 420 kB/s | 18 kB 00:00 2026-02-25T08:17:13.739 INFO:teuthology.orchestra.run.trial066.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 548 kB/s | 23 kB 00:00 2026-02-25T08:17:13.781 INFO:teuthology.orchestra.run.trial066.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 465 kB/s | 20 kB 00:00 2026-02-25T08:17:13.824 INFO:teuthology.orchestra.run.trial066.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 460 kB/s | 19 kB 00:00 2026-02-25T08:17:13.866 INFO:teuthology.orchestra.run.trial066.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 629 kB/s | 26 kB 00:00 2026-02-25T08:17:13.922 INFO:teuthology.orchestra.run.trial066.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 18 MB/s | 1.0 MB 00:00 2026-02-25T08:17:13.965 INFO:teuthology.orchestra.run.trial066.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2026-02-25T08:17:14.007 INFO:teuthology.orchestra.run.trial066.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.0 MB/s | 86 kB 00:00 2026-02-25T08:17:14.050 INFO:teuthology.orchestra.run.trial066.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 1.3 MB/s | 58 kB 00:00 2026-02-25T08:17:14.092 INFO:teuthology.orchestra.run.trial066.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2026-02-25T08:17:14.135 INFO:teuthology.orchestra.run.trial066.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2026-02-25T08:17:14.177 INFO:teuthology.orchestra.run.trial066.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 727 kB/s | 31 kB 00:00 2026-02-25T08:17:14.221 INFO:teuthology.orchestra.run.trial066.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 4.2 MB/s | 188 kB 00:00 2026-02-25T08:17:14.264 INFO:teuthology.orchestra.run.trial066.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2026-02-25T08:17:14.308 INFO:teuthology.orchestra.run.trial066.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 2.8 MB/s | 125 kB 00:00 2026-02-25T08:17:14.350 INFO:teuthology.orchestra.run.trial066.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 843 kB/s | 36 kB 00:00 2026-02-25T08:17:14.392 INFO:teuthology.orchestra.run.trial066.stdout:(90/96): python3-typing-extensions-4.15.0-1.el9 2.0 MB/s | 86 kB 00:00 2026-02-25T08:17:14.436 INFO:teuthology.orchestra.run.trial066.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 2.0 MB/s | 90 kB 00:00 2026-02-25T08:17:14.478 INFO:teuthology.orchestra.run.trial066.stdout:(92/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.1 MB/s | 48 kB 00:00 2026-02-25T08:17:14.520 INFO:teuthology.orchestra.run.trial066.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 524 kB/s | 22 kB 00:00 2026-02-25T08:17:14.563 INFO:teuthology.orchestra.run.trial066.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 472 kB/s | 20 kB 00:00 2026-02-25T08:18:07.278 INFO:teuthology.orchestra.run.trial042.stdout:(95/96): ceph-mgr-dashboard-20.3.0-5338.g6aeb69 353 kB/s | 51 MB 02:26 2026-02-25T08:18:54.667 INFO:teuthology.orchestra.run.trial066.stdout:(95/96): ceph-mgr-dashboard-20.3.0-5338.g6aeb69 242 kB/s | 51 MB 03:34 2026-02-25T08:19:48.649 INFO:teuthology.orchestra.run.trial042.stdout:(96/96): ceph-osd-crimson-20.3.0-5338.g6aeb6975 350 kB/s | 88 MB 04:16 2026-02-25T08:19:48.651 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:19:48.651 INFO:teuthology.orchestra.run.trial042.stdout:Total 885 kB/s | 226 MB 04:20 2026-02-25T08:19:49.021 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:19:49.059 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:19:49.059 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:19:49.858 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:19:49.858 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:19:50.685 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:19:50.686 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-25T08:19:50.690 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-25T08:19:50.711 INFO:teuthology.orchestra.run.trial042.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-25T08:19:50.744 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/96 2026-02-25T08:19:50.750 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-25T08:19:50.752 INFO:teuthology.orchestra.run.trial042.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-25T08:19:50.758 INFO:teuthology.orchestra.run.trial042.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-25T08:19:50.780 INFO:teuthology.orchestra.run.trial042.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-25T08:19:50.793 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-25T08:19:50.796 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-25T08:19:50.802 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-25T08:19:50.804 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-25T08:19:50.815 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-25T08:19:50.821 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-25T08:19:50.845 INFO:teuthology.orchestra.run.trial042.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-25T08:19:50.848 INFO:teuthology.orchestra.run.trial042.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-25T08:19:50.897 INFO:teuthology.orchestra.run.trial042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-25T08:19:50.910 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 18/96 2026-02-25T08:19:50.913 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-25T08:19:50.919 INFO:teuthology.orchestra.run.trial042.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-25T08:19:50.924 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-25T08:19:50.929 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-25T08:19:50.932 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-25T08:19:50.946 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-25T08:19:50.950 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-25T08:19:50.955 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-25T08:19:50.966 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-25T08:19:50.975 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-25T08:19:50.981 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-25T08:19:51.001 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-25T08:19:51.012 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-25T08:19:51.027 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-25T08:19:51.031 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-25T08:19:51.039 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-25T08:19:51.079 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-25T08:19:51.399 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-25T08:19:51.409 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-25T08:19:51.413 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-25T08:19:51.418 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-25T08:19:51.420 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-25T08:19:51.447 INFO:teuthology.orchestra.run.trial042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-25T08:19:51.490 INFO:teuthology.orchestra.run.trial042.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-25T08:19:51.498 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-25T08:19:51.504 INFO:teuthology.orchestra.run.trial042.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-25T08:19:51.507 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-25T08:19:51.513 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-25T08:19:51.516 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-25T08:19:51.523 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-25T08:19:51.526 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-25T08:19:51.554 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-25T08:19:51.564 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-25T08:19:51.569 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-25T08:19:51.576 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-25T08:19:51.610 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-25T08:19:51.832 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-25T08:19:51.855 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-25T08:19:51.859 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-25T08:19:51.917 INFO:teuthology.orchestra.run.trial042.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-25T08:19:51.919 INFO:teuthology.orchestra.run.trial042.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-25T08:19:51.940 INFO:teuthology.orchestra.run.trial042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-25T08:19:52.262 INFO:teuthology.orchestra.run.trial042.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-25T08:19:52.335 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-25T08:19:52.998 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-25T08:19:53.015 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-25T08:19:53.024 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-25T08:19:53.031 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-25T08:19:53.044 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-25T08:19:53.060 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-25T08:19:53.134 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-25T08:19:53.143 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-25T08:19:53.166 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-25T08:19:53.195 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-25T08:19:53.245 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-25T08:19:53.251 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-25T08:19:53.255 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-25T08:19:53.256 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-25T08:19:53.273 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:19:53.273 INFO:teuthology.orchestra.run.trial042.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T08:19:53.273 INFO:teuthology.orchestra.run.trial042.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T08:19:53.273 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:19:53.281 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:19:53.304 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:19:53.305 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T08:19:53.305 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:19:53.323 INFO:teuthology.orchestra.run.trial066.stdout:(96/96): ceph-osd-crimson-20.3.0-5338.g6aeb6975 326 kB/s | 88 MB 04:35 2026-02-25T08:19:53.326 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:19:53.326 INFO:teuthology.orchestra.run.trial066.stdout:Total 813 kB/s | 226 MB 04:44 2026-02-25T08:19:53.338 INFO:teuthology.orchestra.run.trial042.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-25T08:19:53.695 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:19:53.732 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:19:53.733 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:19:53.936 INFO:teuthology.orchestra.run.trial042.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-25T08:19:54.534 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:19:54.535 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:19:55.364 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:19:55.366 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-25T08:19:55.370 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-25T08:19:55.391 INFO:teuthology.orchestra.run.trial066.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-25T08:19:55.425 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/96 2026-02-25T08:19:55.431 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-25T08:19:55.434 INFO:teuthology.orchestra.run.trial066.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-25T08:19:55.438 INFO:teuthology.orchestra.run.trial066.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-25T08:19:55.462 INFO:teuthology.orchestra.run.trial066.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-25T08:19:55.475 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-25T08:19:55.478 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-25T08:19:55.484 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-25T08:19:55.486 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-25T08:19:55.497 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-25T08:19:55.503 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-25T08:19:55.527 INFO:teuthology.orchestra.run.trial066.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-25T08:19:55.530 INFO:teuthology.orchestra.run.trial066.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-25T08:19:55.581 INFO:teuthology.orchestra.run.trial066.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-25T08:19:55.593 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 18/96 2026-02-25T08:19:55.596 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-25T08:19:55.600 INFO:teuthology.orchestra.run.trial066.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-25T08:19:55.606 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-25T08:19:55.610 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-25T08:19:55.613 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-25T08:19:55.628 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-25T08:19:55.632 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-25T08:19:55.638 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-25T08:19:55.649 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-25T08:19:55.658 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-25T08:19:55.664 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-25T08:19:55.684 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-25T08:19:55.695 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-25T08:19:55.710 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-25T08:19:55.715 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-25T08:19:55.723 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-25T08:19:55.762 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-25T08:19:56.087 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-25T08:19:56.097 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-25T08:19:56.101 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-25T08:19:56.106 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-25T08:19:56.108 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-25T08:19:56.135 INFO:teuthology.orchestra.run.trial066.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-25T08:19:56.179 INFO:teuthology.orchestra.run.trial066.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-25T08:19:56.188 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-25T08:19:56.193 INFO:teuthology.orchestra.run.trial066.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-25T08:19:56.197 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-25T08:19:56.202 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-25T08:19:56.206 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-25T08:19:56.213 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-25T08:19:56.216 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-25T08:19:56.244 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-25T08:19:56.253 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-25T08:19:56.259 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-25T08:19:56.266 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-25T08:19:56.300 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-25T08:19:56.524 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-25T08:19:56.547 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-25T08:19:56.551 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-25T08:19:56.609 INFO:teuthology.orchestra.run.trial066.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-25T08:19:56.611 INFO:teuthology.orchestra.run.trial066.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-25T08:19:56.632 INFO:teuthology.orchestra.run.trial066.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-25T08:19:56.957 INFO:teuthology.orchestra.run.trial066.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-25T08:19:57.029 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-25T08:19:57.698 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-25T08:19:57.714 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-25T08:19:57.724 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-25T08:19:57.731 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-25T08:19:57.744 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-25T08:19:57.760 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-25T08:19:57.836 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-25T08:19:57.844 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-25T08:19:57.868 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-25T08:19:57.897 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-25T08:19:57.948 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-25T08:19:57.954 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-25T08:19:57.957 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-25T08:19:57.959 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-25T08:19:57.975 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:19:57.975 INFO:teuthology.orchestra.run.trial066.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T08:19:57.976 INFO:teuthology.orchestra.run.trial066.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T08:19:57.976 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:19:57.983 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:19:58.007 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:19:58.007 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T08:19:58.008 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:19:58.041 INFO:teuthology.orchestra.run.trial066.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-25T08:19:58.627 INFO:teuthology.orchestra.run.trial066.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-25T08:19:59.425 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 80/96 2026-02-25T08:19:59.488 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 81/96 2026-02-25T08:19:59.490 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:19:59.505 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:19:59.505 INFO:teuthology.orchestra.run.trial042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:19:59.505 INFO:teuthology.orchestra.run.trial042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T08:19:59.505 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:19:59.518 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:19:59.542 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:19:59.542 INFO:teuthology.orchestra.run.trial042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:19:59.542 INFO:teuthology.orchestra.run.trial042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T08:19:59.542 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:19:59.542 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:19:59.542 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:19:59.598 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:19:59.600 INFO:teuthology.orchestra.run.trial042.stdout: Installing : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:19:59.610 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 85/96 2026-02-25T08:19:59.634 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 86/96 2026-02-25T08:19:59.636 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:20:02.567 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:20:02.575 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:20:03.041 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:20:03.042 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:20:03.055 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:20:03.056 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:20:03.115 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:20:03.156 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 91/96 2026-02-25T08:20:03.157 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:20:03.178 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:20:03.178 INFO:teuthology.orchestra.run.trial042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:03.178 INFO:teuthology.orchestra.run.trial042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T08:20:03.178 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:20:03.178 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:20:03.178 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:03.189 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:20:03.407 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:20:03.408 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:20:03.429 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:20:03.429 INFO:teuthology.orchestra.run.trial042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:03.429 INFO:teuthology.orchestra.run.trial042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T08:20:03.429 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:20:03.430 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:20:03.430 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:03.513 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:20:03.533 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:20:03.533 INFO:teuthology.orchestra.run.trial042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:03.533 INFO:teuthology.orchestra.run.trial042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T08:20:03.534 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:20:03.534 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:20:03.534 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:03.534 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:20:03.540 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:20:03.565 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:20:04.021 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 80/96 2026-02-25T08:20:04.085 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 81/96 2026-02-25T08:20:04.087 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:20:04.119 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:20:04.119 INFO:teuthology.orchestra.run.trial066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:04.119 INFO:teuthology.orchestra.run.trial066.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T08:20:04.119 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:04.129 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:20:04.152 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:20:04.152 INFO:teuthology.orchestra.run.trial066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:04.152 INFO:teuthology.orchestra.run.trial066.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T08:20:04.152 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:20:04.152 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:20:04.153 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:04.209 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:20:04.211 INFO:teuthology.orchestra.run.trial066.stdout: Installing : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:20:04.220 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 85/96 2026-02-25T08:20:04.244 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 86/96 2026-02-25T08:20:04.245 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:20:04.943 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:20:04.943 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/96 2026-02-25T08:20:04.943 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 5/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 7/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 8/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 9/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 10/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 11/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 12/96 2026-02-25T08:20:04.944 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 13/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 14/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 15/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 16/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 17/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-25T08:20:04.945 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-25T08:20:04.946 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-25T08:20:04.947 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 50/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-25T08:20:04.948 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-25T08:20:04.949 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-25T08:20:04.950 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-25T08:20:04.951 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-25T08:20:04.951 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-25T08:20:04.951 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-25T08:20:04.951 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-25T08:20:04.951 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-25T08:20:04.951 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-25T08:20:05.011 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-25T08:20:05.011 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:05.011 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:20:05.011 INFO:teuthology.orchestra.run.trial042.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:05.012 INFO:teuthology.orchestra.run.trial042.stdout: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T08:20:05.013 INFO:teuthology.orchestra.run.trial042.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T08:20:05.014 INFO:teuthology.orchestra.run.trial042.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T08:20:05.015 INFO:teuthology.orchestra.run.trial042.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T08:20:05.016 INFO:teuthology.orchestra.run.trial042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T08:20:05.017 INFO:teuthology.orchestra.run.trial042.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T08:20:05.018 INFO:teuthology.orchestra.run.trial042.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: zip-3.0-35.el9.x86_64 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:05.019 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:05.080 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-base 2026-02-25T08:20:05.406 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:17 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:05.471 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:05.490 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:05.491 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:05.491 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:05.512 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install cephadm 2026-02-25T08:20:05.829 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:17 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:05.894 INFO:teuthology.orchestra.run.trial042.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:05.912 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:05.913 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:05.913 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:05.933 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:20:06.250 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:18 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:06.335 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:06.335 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:06.336 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repo Size 2026-02-25T08:20:06.336 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:06.336 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:20:06.336 INFO:teuthology.orchestra.run.trial042.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 149 k 2026-02-25T08:20:06.336 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:06.336 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:20:06.336 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:06.337 INFO:teuthology.orchestra.run.trial042.stdout:Install 1 Package 2026-02-25T08:20:06.337 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:06.337 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 149 k 2026-02-25T08:20:06.337 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 455 k 2026-02-25T08:20:06.337 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:20:07.157 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:20:07.163 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:20:07.182 INFO:teuthology.orchestra.run.trial042.stdout:ceph-immutable-object-cache-20.3.0-5338.g6aeb69 176 kB/s | 149 kB 00:00 2026-02-25T08:20:07.183 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:07.183 INFO:teuthology.orchestra.run.trial042.stdout:Total 176 kB/s | 149 kB 00:00 2026-02-25T08:20:07.184 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:20:07.188 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:20:07.188 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:20:07.219 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:20:07.219 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:20:07.275 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:20:07.276 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:20:07.299 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:20:07.299 INFO:teuthology.orchestra.run.trial042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:07.299 INFO:teuthology.orchestra.run.trial042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T08:20:07.299 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:07.614 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:20:07.615 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:20:07.635 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:20:07.636 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:20:07.638 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:20:07.638 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:07.638 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:20:07.638 INFO:teuthology.orchestra.run.trial042.stdout: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:07.638 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:07.638 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:07.665 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr 2026-02-25T08:20:07.696 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:20:07.736 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 91/96 2026-02-25T08:20:07.737 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:20:07.758 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:20:07.758 INFO:teuthology.orchestra.run.trial066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:07.758 INFO:teuthology.orchestra.run.trial066.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T08:20:07.758 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:20:07.758 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:20:07.759 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:07.770 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:20:07.984 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:19 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:07.986 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:20:07.988 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:20:08.009 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:20:08.009 INFO:teuthology.orchestra.run.trial066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:08.009 INFO:teuthology.orchestra.run.trial066.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T08:20:08.009 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:20:08.009 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:20:08.009 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:08.049 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:08.068 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:08.069 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:08.069 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:08.090 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:20:08.091 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:20:08.110 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:20:08.110 INFO:teuthology.orchestra.run.trial066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:08.110 INFO:teuthology.orchestra.run.trial066.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T08:20:08.110 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:20:08.111 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:20:08.111 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:08.111 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:20:08.117 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:20:08.141 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:20:08.411 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:20 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:08.476 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:08.495 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:08.496 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:08.496 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:08.517 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:20:08.831 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:20 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:08.898 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:08.917 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:08.917 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:08.918 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:08.939 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:20:09.259 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:21 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:09.324 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:09.342 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:09.343 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:09.343 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:09.365 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:20:09.556 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:20:09.556 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 5/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 7/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 8/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 9/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 10/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 11/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 12/96 2026-02-25T08:20:09.557 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 13/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 14/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 15/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 16/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 17/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-25T08:20:09.558 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-25T08:20:09.559 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 50/96 2026-02-25T08:20:09.560 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-25T08:20:09.561 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-25T08:20:09.562 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-25T08:20:09.563 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-25T08:20:09.564 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-25T08:20:09.564 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-25T08:20:09.564 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-25T08:20:09.564 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-25T08:20:09.564 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-25T08:20:09.626 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.627 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T08:20:09.628 INFO:teuthology.orchestra.run.trial066.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T08:20:09.629 INFO:teuthology.orchestra.run.trial066.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T08:20:09.630 INFO:teuthology.orchestra.run.trial066.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T08:20:09.631 INFO:teuthology.orchestra.run.trial066.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T08:20:09.632 INFO:teuthology.orchestra.run.trial066.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T08:20:09.633 INFO:teuthology.orchestra.run.trial066.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: zip-3.0-35.el9.x86_64 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:09.634 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:09.680 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:21 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:09.693 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-base 2026-02-25T08:20:09.746 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:09.765 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:09.766 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:09.766 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:09.787 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-osd 2026-02-25T08:20:10.013 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:21 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:10.081 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:10.099 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:10.100 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:10.100 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:10.103 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:22 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:10.122 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install cephadm 2026-02-25T08:20:10.168 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:10.187 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:10.188 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:10.188 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:10.209 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-osd-classic 2026-02-25T08:20:10.441 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:21 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:10.506 INFO:teuthology.orchestra.run.trial066.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:10.523 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:22 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:10.525 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:10.526 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:10.526 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:10.547 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:20:10.588 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:10.606 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:10.607 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:10.607 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:10.628 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-fuse 2026-02-25T08:20:10.861 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:21 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:10.944 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:22 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:10.945 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repo Size 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 149 k 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout:Install 1 Package 2026-02-25T08:20:10.946 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:10.947 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 149 k 2026-02-25T08:20:10.947 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 455 k 2026-02-25T08:20:10.947 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:20:11.029 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repo Size 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout: ceph-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 914 k 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout:Install 1 Package 2026-02-25T08:20:11.030 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:11.031 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 914 k 2026-02-25T08:20:11.031 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 2.9 M 2026-02-25T08:20:11.031 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:20:11.744 INFO:teuthology.orchestra.run.trial066.stdout:ceph-immutable-object-cache-20.3.0-5338.g6aeb69 187 kB/s | 149 kB 00:00 2026-02-25T08:20:11.746 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:11.746 INFO:teuthology.orchestra.run.trial066.stdout:Total 187 kB/s | 149 kB 00:00 2026-02-25T08:20:11.746 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:20:11.751 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:20:11.751 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:20:11.782 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:20:11.783 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:20:11.839 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:20:11.840 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:20:11.862 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:20:11.862 INFO:teuthology.orchestra.run.trial066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:11.862 INFO:teuthology.orchestra.run.trial066.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T08:20:11.862 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:12.172 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:20:12.173 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:12.173 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:20:12.173 INFO:teuthology.orchestra.run.trial066.stdout: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:12.173 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:12.173 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:12.200 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr 2026-02-25T08:20:12.520 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:23 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:12.586 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:12.604 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:12.605 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:12.605 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:12.626 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:20:12.941 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:23 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:13.005 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:13.024 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:13.025 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:13.025 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:13.045 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:20:13.364 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:24 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:13.429 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:13.448 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:13.449 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:13.449 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:13.470 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:20:13.786 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:24 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:13.850 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:13.869 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:13.870 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:13.870 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:13.890 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:20:14.200 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:25 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:14.266 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:14.284 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:14.285 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:14.285 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:14.306 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-osd 2026-02-25T08:20:14.555 INFO:teuthology.orchestra.run.trial042.stdout:ceph-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 259 kB/s | 914 kB 00:03 2026-02-25T08:20:14.556 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:14.556 INFO:teuthology.orchestra.run.trial042.stdout:Total 259 kB/s | 914 kB 00:03 2026-02-25T08:20:14.556 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:20:14.561 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:20:14.561 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:20:14.594 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:20:14.594 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:20:14.618 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:25 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:14.683 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:14.686 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:20:14.702 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:14.702 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:14.703 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:14.704 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:14.724 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-osd-classic 2026-02-25T08:20:14.944 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:15.005 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:15.005 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:15.006 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:20:15.006 INFO:teuthology.orchestra.run.trial042.stdout: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:15.006 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:15.006 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:15.035 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-volume 2026-02-25T08:20:15.036 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:26 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:15.100 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:15.119 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:15.119 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:15.120 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:15.140 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-fuse 2026-02-25T08:20:15.353 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:27 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:15.418 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:15.437 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:15.438 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:15.438 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:15.454 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:26 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:15.460 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install librados-devel 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repo Size 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout: ceph-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 914 k 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:20:15.539 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:15.540 INFO:teuthology.orchestra.run.trial066.stdout:Install 1 Package 2026-02-25T08:20:15.540 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:15.540 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 914 k 2026-02-25T08:20:15.540 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 2.9 M 2026-02-25T08:20:15.540 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:20:15.773 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:27 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:15.858 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repo Size 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout: librados-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 124 k 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:15.859 INFO:teuthology.orchestra.run.trial042.stdout:Install 1 Package 2026-02-25T08:20:15.860 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:15.860 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 124 k 2026-02-25T08:20:15.860 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 434 k 2026-02-25T08:20:15.860 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:20:16.879 INFO:teuthology.orchestra.run.trial066.stdout:ceph-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 682 kB/s | 914 kB 00:01 2026-02-25T08:20:16.880 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:16.880 INFO:teuthology.orchestra.run.trial066.stdout:Total 681 kB/s | 914 kB 00:01 2026-02-25T08:20:16.880 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:20:16.884 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:20:16.884 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:20:16.918 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:20:16.918 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:20:17.008 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:20:17.026 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:17.267 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:17.330 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:17.330 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:17.330 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:20:17.330 INFO:teuthology.orchestra.run.trial066.stdout: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:17.330 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:17.330 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:17.358 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-volume 2026-02-25T08:20:17.433 INFO:teuthology.orchestra.run.trial042.stdout:librados-devel-20.3.0-5338.g6aeb6975.el9.x86_64 78 kB/s | 124 kB 00:01 2026-02-25T08:20:17.434 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:17.434 INFO:teuthology.orchestra.run.trial042.stdout:Total 78 kB/s | 124 kB 00:01 2026-02-25T08:20:17.434 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:20:17.437 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:20:17.437 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:20:17.468 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:20:17.469 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:20:17.523 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:20:17.540 INFO:teuthology.orchestra.run.trial042.stdout: Installing : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:17.627 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:17.679 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:28 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:17.691 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:17.691 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:17.691 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:20:17.691 INFO:teuthology.orchestra.run.trial042.stdout: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:17.691 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:17.691 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:17.719 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install libcephfs2 2026-02-25T08:20:17.746 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:20:17.765 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:17.766 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:17.766 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:17.788 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install librados-devel 2026-02-25T08:20:18.035 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:30 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:18.100 INFO:teuthology.orchestra.run.trial042.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:18.100 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:29 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:18.118 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:18.119 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:18.119 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:18.141 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install libcephfs-devel 2026-02-25T08:20:18.184 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:18.185 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:18.185 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repo Size 2026-02-25T08:20:18.185 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:18.185 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:20:18.185 INFO:teuthology.orchestra.run.trial066.stdout: librados-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 124 k 2026-02-25T08:20:18.185 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:18.185 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:20:18.186 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:18.186 INFO:teuthology.orchestra.run.trial066.stdout:Install 1 Package 2026-02-25T08:20:18.186 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:18.186 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 124 k 2026-02-25T08:20:18.186 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 434 k 2026-02-25T08:20:18.186 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:20:18.458 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:30 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:18.542 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repo Size 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout: libcephfs-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout:Install 1 Package 2026-02-25T08:20:18.543 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:18.544 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 37 k 2026-02-25T08:20:18.544 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 171 k 2026-02-25T08:20:18.544 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:20:18.900 INFO:teuthology.orchestra.run.trial042.stdout:libcephfs-devel-20.3.0-5338.g6aeb6975.el9.x86_6 104 kB/s | 37 kB 00:00 2026-02-25T08:20:18.901 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:18.901 INFO:teuthology.orchestra.run.trial042.stdout:Total 104 kB/s | 37 kB 00:00 2026-02-25T08:20:18.901 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:20:18.904 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:20:18.904 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:20:18.907 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:20:18.907 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:20:18.930 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:20:18.947 INFO:teuthology.orchestra.run.trial042.stdout: Installing : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:19.007 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:19.068 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:19.069 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:19.069 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:20:19.069 INFO:teuthology.orchestra.run.trial042.stdout: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:19.069 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:19.069 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:19.095 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install librados2 2026-02-25T08:20:19.310 INFO:teuthology.orchestra.run.trial066.stdout:librados-devel-20.3.0-5338.g6aeb6975.el9.x86_64 110 kB/s | 124 kB 00:01 2026-02-25T08:20:19.311 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:19.311 INFO:teuthology.orchestra.run.trial066.stdout:Total 110 kB/s | 124 kB 00:01 2026-02-25T08:20:19.311 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:20:19.313 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:20:19.314 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:20:19.345 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:20:19.345 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:20:19.401 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:20:19.416 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:31 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:19.418 INFO:teuthology.orchestra.run.trial066.stdout: Installing : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:19.483 INFO:teuthology.orchestra.run.trial042.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:19.501 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:19.502 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:19.502 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:19.509 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:19.523 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install librbd1 2026-02-25T08:20:19.573 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:19.573 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:19.574 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:20:19.574 INFO:teuthology.orchestra.run.trial066.stdout: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:19.574 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:19.574 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:19.601 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install libcephfs2 2026-02-25T08:20:19.843 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:31 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:19.908 INFO:teuthology.orchestra.run.trial042.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:19.917 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:30 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:19.926 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:19.927 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:19.927 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:19.948 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install python3-rados 2026-02-25T08:20:19.982 INFO:teuthology.orchestra.run.trial066.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:20.004 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:20.004 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:20.004 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:20.023 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install libcephfs-devel 2026-02-25T08:20:20.262 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:32 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:20.327 INFO:teuthology.orchestra.run.trial042.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:20.338 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:31 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:20.344 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:20.345 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:20.345 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:20.367 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install python3-rgw 2026-02-25T08:20:20.421 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:20.422 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:20.422 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repo Size 2026-02-25T08:20:20.422 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout:Install 1 Package 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 37 k 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 171 k 2026-02-25T08:20:20.423 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:20:20.679 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:32 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:20.744 INFO:teuthology.orchestra.run.trial042.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:20.761 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:20.762 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:20.762 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:20.783 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install python3-cephfs 2026-02-25T08:20:20.784 INFO:teuthology.orchestra.run.trial066.stdout:libcephfs-devel-20.3.0-5338.g6aeb6975.el9.x86_6 103 kB/s | 37 kB 00:00 2026-02-25T08:20:20.784 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:20.785 INFO:teuthology.orchestra.run.trial066.stdout:Total 103 kB/s | 37 kB 00:00 2026-02-25T08:20:20.785 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:20:20.787 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:20:20.787 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:20:20.790 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:20:20.790 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:20:20.812 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:20:20.829 INFO:teuthology.orchestra.run.trial066.stdout: Installing : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:20.890 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:20.952 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:20.952 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:20.953 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:20:20.953 INFO:teuthology.orchestra.run.trial066.stdout: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:20.953 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:20.953 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:20.979 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install librados2 2026-02-25T08:20:21.102 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:33 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:21.167 INFO:teuthology.orchestra.run.trial042.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:21.185 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:21.185 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:21.186 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:21.207 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install python3-rbd 2026-02-25T08:20:21.297 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:32 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:21.363 INFO:teuthology.orchestra.run.trial066.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:21.381 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:21.382 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:21.382 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:21.404 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install librbd1 2026-02-25T08:20:21.523 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:33 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:21.588 INFO:teuthology.orchestra.run.trial042.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:21.606 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:21.606 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:21.606 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:21.627 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install rbd-fuse 2026-02-25T08:20:21.721 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:32 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:21.787 INFO:teuthology.orchestra.run.trial066.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:21.804 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:21.805 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:21.805 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:21.826 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install python3-rados 2026-02-25T08:20:21.948 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:33 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:22.031 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:22.032 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:22.032 INFO:teuthology.orchestra.run.trial042.stdout: Package Architecture Version Repository Size 2026-02-25T08:20:22.032 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:22.032 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:20:22.032 INFO:teuthology.orchestra.run.trial042.stdout: rbd-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 86 k 2026-02-25T08:20:22.033 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:22.033 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:20:22.033 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:22.033 INFO:teuthology.orchestra.run.trial042.stdout:Install 1 Package 2026-02-25T08:20:22.033 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:22.033 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 86 k 2026-02-25T08:20:22.033 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 236 k 2026-02-25T08:20:22.033 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:20:22.144 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:33 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:22.210 INFO:teuthology.orchestra.run.trial066.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:22.227 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:22.228 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:22.228 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:22.250 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install python3-rgw 2026-02-25T08:20:22.531 INFO:teuthology.orchestra.run.trial042.stdout:rbd-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 173 kB/s | 86 kB 00:00 2026-02-25T08:20:22.532 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:22.532 INFO:teuthology.orchestra.run.trial042.stdout:Total 172 kB/s | 86 kB 00:00 2026-02-25T08:20:22.532 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:20:22.536 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:20:22.537 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:20:22.567 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:20:22.568 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:20:22.568 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:33 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:22.620 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:20:22.633 INFO:teuthology.orchestra.run.trial066.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:22.637 INFO:teuthology.orchestra.run.trial042.stdout: Installing : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:22.651 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:22.651 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:22.651 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:22.674 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install python3-cephfs 2026-02-25T08:20:22.720 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:22.782 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:22.782 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:22.782 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:20:22.783 INFO:teuthology.orchestra.run.trial042.stdout: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:22.783 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:22.783 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:22.810 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install rbd-mirror 2026-02-25T08:20:22.992 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:33 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:23.058 INFO:teuthology.orchestra.run.trial066.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:23.076 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:23.076 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:23.076 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:23.098 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install python3-rbd 2026-02-25T08:20:23.125 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:35 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:23.208 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:23.208 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repo Size 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout: rbd-mirror x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.6 M 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout:Install 1 Package 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:23.209 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 2.6 M 2026-02-25T08:20:23.210 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 11 M 2026-02-25T08:20:23.210 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:20:23.413 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:34 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:23.478 INFO:teuthology.orchestra.run.trial066.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:23.496 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:23.497 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:23.497 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:23.519 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install rbd-fuse 2026-02-25T08:20:23.841 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:34 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:23.925 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout: Package Architecture Version Repository Size 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout: rbd-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 86 k 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout:Install 1 Package 2026-02-25T08:20:23.926 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:23.927 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 86 k 2026-02-25T08:20:23.927 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 236 k 2026-02-25T08:20:23.927 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:20:24.424 INFO:teuthology.orchestra.run.trial066.stdout:rbd-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 173 kB/s | 86 kB 00:00 2026-02-25T08:20:24.424 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:24.424 INFO:teuthology.orchestra.run.trial066.stdout:Total 173 kB/s | 86 kB 00:00 2026-02-25T08:20:24.424 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:20:24.429 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:20:24.429 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:20:24.460 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:20:24.460 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:20:24.512 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:20:24.529 INFO:teuthology.orchestra.run.trial066.stdout: Installing : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:24.615 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:24.678 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:24.679 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:24.679 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:20:24.679 INFO:teuthology.orchestra.run.trial066.stdout: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:24.679 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:24.679 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:24.707 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install rbd-mirror 2026-02-25T08:20:25.026 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:36 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:25.109 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:25.109 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repo Size 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout: rbd-mirror x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.6 M 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout:Install 1 Package 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:25.110 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 2.6 M 2026-02-25T08:20:25.111 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 11 M 2026-02-25T08:20:25.111 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:20:29.031 INFO:teuthology.orchestra.run.trial042.stdout:rbd-mirror-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 456 kB/s | 2.6 MB 00:05 2026-02-25T08:20:29.031 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:29.031 INFO:teuthology.orchestra.run.trial042.stdout:Total 456 kB/s | 2.6 MB 00:05 2026-02-25T08:20:29.031 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:20:29.036 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:20:29.036 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:20:29.075 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:20:29.075 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:20:29.242 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:20:29.243 INFO:teuthology.orchestra.run.trial042.stdout: Installing : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:29.275 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:29.275 INFO:teuthology.orchestra.run.trial042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:29.275 INFO:teuthology.orchestra.run.trial042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T08:20:29.275 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:20:29.275 INFO:teuthology.orchestra.run.trial042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:20:29.276 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:29.612 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:29.612 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:29.612 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:20:29.613 INFO:teuthology.orchestra.run.trial042.stdout: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:29.613 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:29.613 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:29.641 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install rbd-nbd 2026-02-25T08:20:29.961 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:41 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:30.045 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:30.046 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout: Package Architecture Version Repository Size 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout: rbd-nbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 166 k 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout:Install 1 Package 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 166 k 2026-02-25T08:20:30.047 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 496 k 2026-02-25T08:20:30.048 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:20:30.684 INFO:teuthology.orchestra.run.trial042.stdout:rbd-nbd-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 261 kB/s | 166 kB 00:00 2026-02-25T08:20:30.684 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:30.685 INFO:teuthology.orchestra.run.trial042.stdout:Total 261 kB/s | 166 kB 00:00 2026-02-25T08:20:30.685 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:20:30.689 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:20:30.689 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:20:30.720 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:20:30.720 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:20:30.777 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:20:30.794 INFO:teuthology.orchestra.run.trial042.stdout: Installing : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:30.881 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:30.943 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:30.943 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:30.943 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:20:30.944 INFO:teuthology.orchestra.run.trial042.stdout: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:30.944 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:30.944 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:30.972 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-osd-crimson 2026-02-25T08:20:31.289 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:43 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:31.355 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:31.372 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:31.373 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:20:31.373 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:20:31.395 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-25T08:20:31.631 INFO:teuthology.orchestra.run.trial066.stdout:rbd-mirror-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 407 kB/s | 2.6 MB 00:06 2026-02-25T08:20:31.633 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:31.633 INFO:teuthology.orchestra.run.trial066.stdout:Total 407 kB/s | 2.6 MB 00:06 2026-02-25T08:20:31.633 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:20:31.639 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:20:31.639 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:20:31.677 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:20:31.677 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:20:31.712 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:11:43 ago on Wed 25 Feb 2026 08:08:48 AM UTC. 2026-02-25T08:20:31.798 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout: Package Arch Version Repo Size 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout:Installing: 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 433 M 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout:Installing weak dependencies: 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout: ceph-debugsource x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 12 M 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout:Transaction Summary 2026-02-25T08:20:31.799 INFO:teuthology.orchestra.run.trial042.stdout:================================================================================ 2026-02-25T08:20:31.800 INFO:teuthology.orchestra.run.trial042.stdout:Install 2 Packages 2026-02-25T08:20:31.800 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:20:31.800 INFO:teuthology.orchestra.run.trial042.stdout:Total download size: 445 M 2026-02-25T08:20:31.800 INFO:teuthology.orchestra.run.trial042.stdout:Installed size: 2.7 G 2026-02-25T08:20:31.800 INFO:teuthology.orchestra.run.trial042.stdout:Downloading Packages: 2026-02-25T08:20:31.848 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:20:31.850 INFO:teuthology.orchestra.run.trial066.stdout: Installing : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:31.881 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:31.882 INFO:teuthology.orchestra.run.trial066.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:20:31.882 INFO:teuthology.orchestra.run.trial066.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T08:20:31.882 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:20:31.882 INFO:teuthology.orchestra.run.trial066.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:20:31.882 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:32.218 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:32.218 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:32.219 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:20:32.219 INFO:teuthology.orchestra.run.trial066.stdout: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:32.219 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:32.219 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:32.245 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install rbd-nbd 2026-02-25T08:20:32.569 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:43 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:32.654 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout: Package Architecture Version Repository Size 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout: rbd-nbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 166 k 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout:Install 1 Package 2026-02-25T08:20:32.655 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:32.656 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 166 k 2026-02-25T08:20:32.656 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 496 k 2026-02-25T08:20:32.656 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:20:33.289 INFO:teuthology.orchestra.run.trial066.stdout:rbd-nbd-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 262 kB/s | 166 kB 00:00 2026-02-25T08:20:33.290 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:20:33.290 INFO:teuthology.orchestra.run.trial066.stdout:Total 262 kB/s | 166 kB 00:00 2026-02-25T08:20:33.290 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:20:33.295 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:20:33.295 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:20:33.326 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:20:33.327 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:20:33.383 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:20:33.400 INFO:teuthology.orchestra.run.trial066.stdout: Installing : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:33.489 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:33.553 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:20:33.553 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:33.553 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:20:33.553 INFO:teuthology.orchestra.run.trial066.stdout: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:20:33.553 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:33.553 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:33.581 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-osd-crimson 2026-02-25T08:20:33.901 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:44 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:33.966 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:20:33.984 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:33.984 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:20:33.985 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:20:34.007 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-25T08:20:34.323 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:11:45 ago on Wed 25 Feb 2026 08:08:49 AM UTC. 2026-02-25T08:20:34.406 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:20:34.407 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:34.407 INFO:teuthology.orchestra.run.trial066.stdout: Package Arch Version Repo Size 2026-02-25T08:20:34.407 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:34.407 INFO:teuthology.orchestra.run.trial066.stdout:Installing: 2026-02-25T08:20:34.407 INFO:teuthology.orchestra.run.trial066.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 433 M 2026-02-25T08:20:34.407 INFO:teuthology.orchestra.run.trial066.stdout:Installing weak dependencies: 2026-02-25T08:20:34.407 INFO:teuthology.orchestra.run.trial066.stdout: ceph-debugsource x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 12 M 2026-02-25T08:20:34.407 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:34.408 INFO:teuthology.orchestra.run.trial066.stdout:Transaction Summary 2026-02-25T08:20:34.408 INFO:teuthology.orchestra.run.trial066.stdout:================================================================================ 2026-02-25T08:20:34.408 INFO:teuthology.orchestra.run.trial066.stdout:Install 2 Packages 2026-02-25T08:20:34.408 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:20:34.408 INFO:teuthology.orchestra.run.trial066.stdout:Total download size: 445 M 2026-02-25T08:20:34.408 INFO:teuthology.orchestra.run.trial066.stdout:Installed size: 2.7 G 2026-02-25T08:20:34.408 INFO:teuthology.orchestra.run.trial066.stdout:Downloading Packages: 2026-02-25T08:21:05.166 INFO:teuthology.orchestra.run.trial042.stdout:(1/2): ceph-debugsource-20.3.0-5338.g6aeb6975.e 370 kB/s | 12 MB 00:33 2026-02-25T08:21:05.333 INFO:teuthology.orchestra.run.trial066.stdout:(1/2): ceph-debugsource-20.3.0-5338.g6aeb6975.e 399 kB/s | 12 MB 00:30 2026-02-25T08:38:50.143 INFO:teuthology.orchestra.run.trial042.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5338.g 403 kB/s | 433 MB 18:18 2026-02-25T08:38:50.144 INFO:teuthology.orchestra.run.trial042.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:38:50.144 INFO:teuthology.orchestra.run.trial042.stdout:Total 414 kB/s | 445 MB 18:18 2026-02-25T08:38:50.145 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction check 2026-02-25T08:38:50.150 INFO:teuthology.orchestra.run.trial042.stdout:Transaction check succeeded. 2026-02-25T08:38:50.150 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction test 2026-02-25T08:38:51.520 INFO:teuthology.orchestra.run.trial042.stdout:Transaction test succeeded. 2026-02-25T08:38:51.520 INFO:teuthology.orchestra.run.trial042.stdout:Running transaction 2026-02-25T08:38:55.105 INFO:teuthology.orchestra.run.trial042.stdout: Preparing : 1/1 2026-02-25T08:39:17.833 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:39:17.851 INFO:teuthology.orchestra.run.trial042.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:39:19.910 INFO:teuthology.orchestra.run.trial042.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:39:19.910 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:39:19.974 INFO:teuthology.orchestra.run.trial042.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:39:19.974 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:39:19.974 INFO:teuthology.orchestra.run.trial042.stdout:Installed: 2026-02-25T08:39:19.974 INFO:teuthology.orchestra.run.trial042.stdout: ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:39:19.975 INFO:teuthology.orchestra.run.trial042.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:39:19.975 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:39:19.975 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:39:20.056 DEBUG:teuthology.parallel:result is None 2026-02-25T08:39:32.935 INFO:teuthology.orchestra.run.trial066.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5338.g 389 kB/s | 433 MB 18:58 2026-02-25T08:39:32.937 INFO:teuthology.orchestra.run.trial066.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:39:32.937 INFO:teuthology.orchestra.run.trial066.stdout:Total 400 kB/s | 445 MB 18:58 2026-02-25T08:39:32.937 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction check 2026-02-25T08:39:32.942 INFO:teuthology.orchestra.run.trial066.stdout:Transaction check succeeded. 2026-02-25T08:39:32.943 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction test 2026-02-25T08:39:34.328 INFO:teuthology.orchestra.run.trial066.stdout:Transaction test succeeded. 2026-02-25T08:39:34.328 INFO:teuthology.orchestra.run.trial066.stdout:Running transaction 2026-02-25T08:39:37.957 INFO:teuthology.orchestra.run.trial066.stdout: Preparing : 1/1 2026-02-25T08:40:00.666 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:40:00.685 INFO:teuthology.orchestra.run.trial066.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:40:02.632 INFO:teuthology.orchestra.run.trial066.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:40:02.632 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:40:02.698 INFO:teuthology.orchestra.run.trial066.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:40:02.698 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:02.698 INFO:teuthology.orchestra.run.trial066.stdout:Installed: 2026-02-25T08:40:02.698 INFO:teuthology.orchestra.run.trial066.stdout: ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:40:02.699 INFO:teuthology.orchestra.run.trial066.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:40:02.699 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:02.699 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:02.778 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:02.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:02.938 DEBUG:teuthology.orchestra.run.trial042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:40:02.959 INFO:teuthology.orchestra.run.trial042.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:02.959 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:02.959 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:40:02.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:03.079 DEBUG:teuthology.orchestra.run.trial066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:40:03.100 INFO:teuthology.orchestra.run.trial066.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:03.100 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:03.100 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:40:03.102 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-25T08:40:03.102 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:03.102 DEBUG:teuthology.orchestra.run.trial042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T08:40:03.130 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:40:03.130 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T08:40:03.166 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-25T08:40:03.167 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:03.167 DEBUG:teuthology.orchestra.run.trial042:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T08:40:03.193 DEBUG:teuthology.orchestra.run.trial042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T08:40:03.257 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:40:03.258 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T08:40:03.284 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T08:40:03.348 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-25T08:40:03.349 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:03.349 DEBUG:teuthology.orchestra.run.trial042:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T08:40:03.373 DEBUG:teuthology.orchestra.run.trial042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T08:40:03.438 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:40:03.438 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T08:40:03.463 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T08:40:03.527 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-25T08:40:03.527 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:03.528 DEBUG:teuthology.orchestra.run.trial042:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T08:40:03.553 DEBUG:teuthology.orchestra.run.trial042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T08:40:03.616 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:40:03.616 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T08:40:03.642 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T08:40:03.705 INFO:teuthology.run_tasks:Running task pexec... 2026-02-25T08:40:03.724 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-25T08:40:03.724 DEBUG:teuthology.orchestra.run.trial042:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-25T08:40:03.725 DEBUG:teuthology.orchestra.run.trial066:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-25T08:40:03.728 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial042.front.sepia.ceph.com 2026-02-25T08:40:03.728 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-25T08:40:03.748 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial066.front.sepia.ceph.com 2026-02-25T08:40:03.748 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-25T08:40:03.779 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:03.798 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:03.798 INFO:teuthology.run_tasks:Running task install... 2026-02-25T08:40:03.813 DEBUG:teuthology.task.install:project ceph 2026-02-25T08:40:03.813 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}} 2026-02-25T08:40:03.813 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'} 2026-02-25T08:40:03.813 INFO:teuthology.task.install:Using flavor: debug 2026-02-25T08:40:03.822 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T08:40:03.822 INFO:teuthology.task.install:extra packages: [] 2026-02-25T08:40:03.822 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:40:03.823 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:03.824 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:40:03.825 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:04.008 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:40:04.008 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:40:04.017 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:40:04.017 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:40:04.287 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:40:04.287 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:04.287 DEBUG:teuthology.orchestra.run.trial042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:40:04.312 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-25T08:40:04.313 DEBUG:teuthology.orchestra.run.trial042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:40:04.325 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:40:04.325 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:40:04.325 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:40:04.350 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-25T08:40:04.350 DEBUG:teuthology.orchestra.run.trial066:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:40:04.377 DEBUG:teuthology.orchestra.run.trial042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:40:04.415 DEBUG:teuthology.orchestra.run.trial066:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:40:04.441 DEBUG:teuthology.orchestra.run.trial042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:40:04.480 DEBUG:teuthology.orchestra.run.trial066:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:40:04.496 INFO:teuthology.orchestra.run.trial042.stdout:check_obsoletes = 1 2026-02-25T08:40:04.506 DEBUG:teuthology.orchestra.run.trial042:> sudo yum clean all 2026-02-25T08:40:04.535 INFO:teuthology.orchestra.run.trial066.stdout:check_obsoletes = 1 2026-02-25T08:40:04.545 DEBUG:teuthology.orchestra.run.trial066:> sudo yum clean all 2026-02-25T08:40:04.667 INFO:teuthology.orchestra.run.trial042.stdout:56 files removed 2026-02-25T08:40:04.683 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-radosgw 2026-02-25T08:40:04.707 INFO:teuthology.orchestra.run.trial066.stdout:56 files removed 2026-02-25T08:40:04.722 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-radosgw 2026-02-25T08:40:05.432 INFO:teuthology.orchestra.run.trial042.stdout:ceph packages for x86_64 158 kB/s | 94 kB 00:00 2026-02-25T08:40:05.464 INFO:teuthology.orchestra.run.trial066.stdout:ceph packages for x86_64 161 kB/s | 94 kB 00:00 2026-02-25T08:40:06.004 INFO:teuthology.orchestra.run.trial042.stdout:ceph noarch packages 45 kB/s | 24 kB 00:00 2026-02-25T08:40:06.038 INFO:teuthology.orchestra.run.trial066.stdout:ceph noarch packages 45 kB/s | 24 kB 00:00 2026-02-25T08:40:06.453 INFO:teuthology.orchestra.run.trial042.stdout:ceph source packages 4.9 kB/s | 2.1 kB 00:00 2026-02-25T08:40:06.474 INFO:teuthology.orchestra.run.trial066.stdout:ceph source packages 5.0 kB/s | 2.1 kB 00:00 2026-02-25T08:40:06.842 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - BaseOS 26 MB/s | 8.9 MB 00:00 2026-02-25T08:40:12.602 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - AppStream 5.2 MB/s | 27 MB 00:05 2026-02-25T08:40:14.917 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-25T08:40:16.176 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - CRB 6.7 MB/s | 8.0 MB 00:01 2026-02-25T08:40:17.089 INFO:teuthology.orchestra.run.trial066.stdout:CentOS Stream 9 - Extras packages 117 kB/s | 20 kB 00:00 2026-02-25T08:40:17.865 INFO:teuthology.orchestra.run.trial066.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2026-02-25T08:40:18.358 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - AppStream 9.3 MB/s | 27 MB 00:02 2026-02-25T08:40:21.127 INFO:teuthology.orchestra.run.trial066.stdout:lab-extras 2.9 MB/s | 50 kB 00:00 2026-02-25T08:40:22.305 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:22.323 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:22.324 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:22.324 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:22.348 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-test 2026-02-25T08:40:22.673 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:22.738 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:22.756 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:22.756 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:22.756 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:22.778 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph 2026-02-25T08:40:23.097 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:23.162 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:23.180 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:23.181 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:23.181 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:23.203 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-base 2026-02-25T08:40:23.526 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:23.591 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:23.608 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:23.609 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:23.609 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:23.631 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install cephadm 2026-02-25T08:40:23.759 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 8.0 MB 00:03 2026-02-25T08:40:23.956 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:24.022 INFO:teuthology.orchestra.run.trial066.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:24.039 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:24.040 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:24.040 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:24.062 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:40:24.383 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:24.448 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:24.466 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:24.466 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:24.467 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:24.488 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr 2026-02-25T08:40:24.661 INFO:teuthology.orchestra.run.trial042.stdout:CentOS Stream 9 - Extras packages 122 kB/s | 20 kB 00:00 2026-02-25T08:40:24.808 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:24.874 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:24.891 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:24.892 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:24.892 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:24.914 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:40:25.233 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:25.297 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:25.315 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:25.316 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:25.316 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:25.338 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:40:25.372 INFO:teuthology.orchestra.run.trial042.stdout:Extra Packages for Enterprise Linux 32 MB/s | 20 MB 00:00 2026-02-25T08:40:25.656 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:25.720 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:25.738 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:25.739 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:25.739 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:25.760 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:40:26.079 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:26.145 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:26.163 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:26.163 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:26.164 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:26.185 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:40:26.502 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:26.568 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:26.586 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:26.587 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:26.587 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:26.609 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-osd 2026-02-25T08:40:26.930 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:26.995 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:27.013 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:27.013 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:27.014 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:27.035 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-osd-classic 2026-02-25T08:40:27.355 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:27.422 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:27.440 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:27.441 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:27.441 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:27.464 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-fuse 2026-02-25T08:40:27.787 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:27.852 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:27.870 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:27.870 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:27.870 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:27.891 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install ceph-volume 2026-02-25T08:40:28.211 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:28.277 INFO:teuthology.orchestra.run.trial066.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:28.298 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:28.298 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:28.299 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:28.319 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install librados-devel 2026-02-25T08:40:28.603 INFO:teuthology.orchestra.run.trial042.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T08:40:28.637 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:28.702 INFO:teuthology.orchestra.run.trial066.stdout:Package librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:28.720 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:28.721 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:28.721 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:28.743 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install libcephfs2 2026-02-25T08:40:29.061 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:29.126 INFO:teuthology.orchestra.run.trial066.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:29.144 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:29.145 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:29.145 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:29.167 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install libcephfs-devel 2026-02-25T08:40:29.490 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:29.558 INFO:teuthology.orchestra.run.trial066.stdout:Package libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:29.576 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:29.576 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:29.576 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:29.599 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install librados2 2026-02-25T08:40:29.774 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:29.792 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:29.793 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:29.793 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:29.816 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-test 2026-02-25T08:40:29.919 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:29.985 INFO:teuthology.orchestra.run.trial066.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:30.003 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:30.003 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:30.003 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:30.025 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install librbd1 2026-02-25T08:40:30.136 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:30.201 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:30.219 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:30.219 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:30.219 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:30.241 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph 2026-02-25T08:40:30.348 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:30.415 INFO:teuthology.orchestra.run.trial066.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:30.433 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:30.433 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:30.434 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:30.455 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install python3-rados 2026-02-25T08:40:30.559 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:30.624 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:30.641 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:30.642 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:30.642 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:30.663 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-base 2026-02-25T08:40:30.776 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:30.843 INFO:teuthology.orchestra.run.trial066.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:30.861 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:30.861 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:30.861 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:30.884 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install python3-rgw 2026-02-25T08:40:30.982 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:31.047 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:31.065 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:31.066 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:31.066 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:31.087 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install cephadm 2026-02-25T08:40:31.204 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:31.270 INFO:teuthology.orchestra.run.trial066.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:31.288 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:31.289 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:31.289 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:31.311 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install python3-cephfs 2026-02-25T08:40:31.407 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:31.473 INFO:teuthology.orchestra.run.trial042.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:31.491 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:31.491 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:31.492 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:31.514 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:40:31.634 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:31.700 INFO:teuthology.orchestra.run.trial066.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:31.718 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:31.718 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:31.719 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:31.741 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install python3-rbd 2026-02-25T08:40:31.832 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:31.898 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:31.916 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:31.917 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:31.917 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:31.939 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr 2026-02-25T08:40:32.068 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:32.134 INFO:teuthology.orchestra.run.trial066.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:32.152 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:32.153 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:32.153 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:32.175 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install rbd-fuse 2026-02-25T08:40:32.263 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:32.329 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:32.347 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:32.347 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:32.348 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:32.369 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:40:32.496 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:32.563 INFO:teuthology.orchestra.run.trial066.stdout:Package rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:32.580 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:32.581 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:32.581 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:32.602 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install rbd-mirror 2026-02-25T08:40:32.694 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:32.760 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:32.778 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:32.778 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:32.778 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:32.800 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:40:32.917 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:32.982 INFO:teuthology.orchestra.run.trial066.stdout:Package rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:33.000 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:33.001 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:33.001 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:33.022 DEBUG:teuthology.orchestra.run.trial066:> sudo yum -y install rbd-nbd 2026-02-25T08:40:33.121 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:33.186 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:33.204 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:33.205 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:33.205 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:33.226 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:40:33.343 INFO:teuthology.orchestra.run.trial066.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:40:21 AM UTC. 2026-02-25T08:40:33.408 INFO:teuthology.orchestra.run.trial066.stdout:Package rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:33.426 INFO:teuthology.orchestra.run.trial066.stdout:Dependencies resolved. 2026-02-25T08:40:33.426 INFO:teuthology.orchestra.run.trial066.stdout:Nothing to do. 2026-02-25T08:40:33.427 INFO:teuthology.orchestra.run.trial066.stdout:Complete! 2026-02-25T08:40:33.450 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:33.541 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:33.607 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:33.624 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:33.625 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:33.625 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:33.647 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:40:33.968 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:34.035 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:34.053 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:34.053 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:34.053 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:34.075 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-osd 2026-02-25T08:40:34.398 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:34.463 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:34.481 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:34.482 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:34.482 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:34.503 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-osd-classic 2026-02-25T08:40:34.813 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:34.878 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:34.895 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:34.896 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:34.896 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:34.918 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-fuse 2026-02-25T08:40:35.239 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:35.304 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:35.322 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:35.322 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:35.322 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:35.344 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install ceph-volume 2026-02-25T08:40:35.663 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:35.730 INFO:teuthology.orchestra.run.trial042.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:35.748 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:35.748 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:35.748 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:35.770 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install librados-devel 2026-02-25T08:40:36.089 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:36.155 INFO:teuthology.orchestra.run.trial042.stdout:Package librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:36.172 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:36.173 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:36.173 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:36.195 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install libcephfs2 2026-02-25T08:40:36.519 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:36.586 INFO:teuthology.orchestra.run.trial042.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:36.603 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:36.604 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:36.604 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:36.626 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install libcephfs-devel 2026-02-25T08:40:36.944 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:37.009 INFO:teuthology.orchestra.run.trial042.stdout:Package libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:37.026 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:37.027 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:37.027 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:37.049 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install librados2 2026-02-25T08:40:37.369 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:37.434 INFO:teuthology.orchestra.run.trial042.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:37.452 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:37.453 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:37.453 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:37.476 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install librbd1 2026-02-25T08:40:37.798 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:37.864 INFO:teuthology.orchestra.run.trial042.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:37.882 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:37.882 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:37.882 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:37.904 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install python3-rados 2026-02-25T08:40:38.224 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:38.289 INFO:teuthology.orchestra.run.trial042.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:38.308 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:38.309 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:38.309 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:38.330 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install python3-rgw 2026-02-25T08:40:38.650 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:38.715 INFO:teuthology.orchestra.run.trial042.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:38.732 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:38.733 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:38.733 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:38.754 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install python3-cephfs 2026-02-25T08:40:39.066 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:39.132 INFO:teuthology.orchestra.run.trial042.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:39.149 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:39.150 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:39.150 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:39.171 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install python3-rbd 2026-02-25T08:40:39.490 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:39.557 INFO:teuthology.orchestra.run.trial042.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:39.575 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:39.575 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:39.575 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:39.597 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install rbd-fuse 2026-02-25T08:40:39.914 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:39.979 INFO:teuthology.orchestra.run.trial042.stdout:Package rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:39.997 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:39.998 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:39.998 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:40.018 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install rbd-mirror 2026-02-25T08:40:40.340 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:40.406 INFO:teuthology.orchestra.run.trial042.stdout:Package rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:40.423 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:40.424 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:40.424 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:40.446 DEBUG:teuthology.orchestra.run.trial042:> sudo yum -y install rbd-nbd 2026-02-25T08:40:40.771 INFO:teuthology.orchestra.run.trial042.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:40:28 AM UTC. 2026-02-25T08:40:40.836 INFO:teuthology.orchestra.run.trial042.stdout:Package rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:40.854 INFO:teuthology.orchestra.run.trial042.stdout:Dependencies resolved. 2026-02-25T08:40:40.855 INFO:teuthology.orchestra.run.trial042.stdout:Nothing to do. 2026-02-25T08:40:40.855 INFO:teuthology.orchestra.run.trial042.stdout:Complete! 2026-02-25T08:40:40.881 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:40.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:41.061 DEBUG:teuthology.orchestra.run.trial042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:40:41.082 INFO:teuthology.orchestra.run.trial042.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:41.082 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:41.083 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:40:41.085 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:41.246 DEBUG:teuthology.orchestra.run.trial066:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:40:41.267 INFO:teuthology.orchestra.run.trial066.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:41.268 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:41.268 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:40:41.270 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-25T08:40:41.271 INFO:teuthology.run_tasks:Running task ceph... 2026-02-25T08:40:41.376 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-25T08:40:41.377 DEBUG:teuthology.orchestra.run.trial042:> sudo chmod 777 /var/log/ceph 2026-02-25T08:40:41.379 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod 777 /var/log/ceph 2026-02-25T08:40:41.404 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-25T08:40:41.404 DEBUG:teuthology.orchestra.run.trial042:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T08:40:41.444 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T08:40:41.469 INFO:tasks.ceph:Creating extra log directories... 2026-02-25T08:40:41.469 DEBUG:teuthology.orchestra.run.trial042:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T08:40:41.509 DEBUG:teuthology.orchestra.run.trial066:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T08:40:41.546 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-25T08:40:41.546 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-25T08:40:41.547 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053', 'branch': 'wip-logstoremanager', 'description': 'crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/admin_socket_objecter_requests}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '69053', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-logstoremanager'}, 'ceph': {'conf': {'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d'}}, 'owner': 'scheduled_matan@soko04.front.sepia.ceph.com', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 869, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d', 'targets': {'trial042.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHMcSn1dMgsIPlpZvacO3/oO2wo8fm3GXxt2upgM4Q2NaRvqxyoADLecnIfr8kxOpLNO6EONXNz4kJMRZREbL9w=', 'trial066.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJTyCjD9GrtQ38KMvpQXYdn4Q/0/aBehuDcfC5KMu3ylLZof83uU8dNjnHr3v4l3qEAERHjqK7Bv0sIjAbDs7Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'cluster': 'ceph'}}, {'thrashosds': {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'chance_trim_stale_osdmaps': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400}}, {'radosbench': {'clients': ['client.0'], 'time': 150}}, {'admin_socket': {'client.0': {'objecter_requests': {'test': 'http://git.ceph.com/?p={repo};a=blob_plain;f=src/test/admin_socket/objecter_requests;hb={branch}'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-25_07:55:35', 'tube': 'trial', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-25T08:40:41.548 DEBUG:teuthology.orchestra.run.trial042:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T08:40:41.571 DEBUG:teuthology.orchestra.run.trial066:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T08:40:41.588 DEBUG:teuthology.orchestra.run.trial042:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T08:40:41.627 DEBUG:teuthology.orchestra.run.trial066:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T08:40:41.650 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:41.650 DEBUG:teuthology.orchestra.run.trial042:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T08:40:41.704 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T08:40:41.704 DEBUG:teuthology.orchestra.run.trial042:> stat /dev/vg_nvme/lv_1 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout:Device: 6h/6d Inode: 1306 Links: 1 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout:Access: 2026-02-25 08:20:29.427996271 +0000 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout:Modify: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout:Change: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:41.759 INFO:teuthology.orchestra.run.trial042.stdout: Birth: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:41.760 DEBUG:teuthology.orchestra.run.trial042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T08:40:41.821 INFO:teuthology.orchestra.run.trial042.stderr:1+0 records in 2026-02-25T08:40:41.821 INFO:teuthology.orchestra.run.trial042.stderr:1+0 records out 2026-02-25T08:40:41.821 INFO:teuthology.orchestra.run.trial042.stderr:512 bytes copied, 0.000157573 s, 3.2 MB/s 2026-02-25T08:40:41.822 DEBUG:teuthology.orchestra.run.trial042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T08:40:41.879 DEBUG:teuthology.orchestra.run.trial042:> stat /dev/vg_nvme/lv_2 2026-02-25T08:40:41.934 INFO:teuthology.orchestra.run.trial042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T08:40:41.935 INFO:teuthology.orchestra.run.trial042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:41.935 INFO:teuthology.orchestra.run.trial042.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-25T08:40:41.935 INFO:teuthology.orchestra.run.trial042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:41.935 INFO:teuthology.orchestra.run.trial042.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:41.935 INFO:teuthology.orchestra.run.trial042.stdout:Access: 2026-02-25 08:20:29.427996271 +0000 2026-02-25T08:40:41.935 INFO:teuthology.orchestra.run.trial042.stdout:Modify: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:41.935 INFO:teuthology.orchestra.run.trial042.stdout:Change: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:41.935 INFO:teuthology.orchestra.run.trial042.stdout: Birth: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:41.935 DEBUG:teuthology.orchestra.run.trial042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T08:40:41.996 INFO:teuthology.orchestra.run.trial042.stderr:1+0 records in 2026-02-25T08:40:41.996 INFO:teuthology.orchestra.run.trial042.stderr:1+0 records out 2026-02-25T08:40:41.996 INFO:teuthology.orchestra.run.trial042.stderr:512 bytes copied, 0.00016155 s, 3.2 MB/s 2026-02-25T08:40:41.997 DEBUG:teuthology.orchestra.run.trial042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T08:40:42.053 DEBUG:teuthology.orchestra.run.trial042:> stat /dev/vg_nvme/lv_3 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout:Access: 2026-02-25 08:20:29.427996271 +0000 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout:Modify: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout:Change: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:42.110 INFO:teuthology.orchestra.run.trial042.stdout: Birth: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:42.110 DEBUG:teuthology.orchestra.run.trial042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T08:40:42.172 INFO:teuthology.orchestra.run.trial042.stderr:1+0 records in 2026-02-25T08:40:42.172 INFO:teuthology.orchestra.run.trial042.stderr:1+0 records out 2026-02-25T08:40:42.172 INFO:teuthology.orchestra.run.trial042.stderr:512 bytes copied, 0.000155619 s, 3.3 MB/s 2026-02-25T08:40:42.173 DEBUG:teuthology.orchestra.run.trial042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T08:40:42.229 DEBUG:teuthology.orchestra.run.trial042:> stat /dev/vg_nvme/lv_4 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout:Device: 6h/6d Inode: 1315 Links: 1 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout:Access: 2026-02-25 08:20:29.428996252 +0000 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout:Modify: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout:Change: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:42.284 INFO:teuthology.orchestra.run.trial042.stdout: Birth: 2026-02-25 08:09:46.464710762 +0000 2026-02-25T08:40:42.284 DEBUG:teuthology.orchestra.run.trial042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T08:40:42.345 INFO:teuthology.orchestra.run.trial042.stderr:1+0 records in 2026-02-25T08:40:42.345 INFO:teuthology.orchestra.run.trial042.stderr:1+0 records out 2026-02-25T08:40:42.345 INFO:teuthology.orchestra.run.trial042.stderr:512 bytes copied, 0.000134821 s, 3.8 MB/s 2026-02-25T08:40:42.346 DEBUG:teuthology.orchestra.run.trial042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T08:40:42.403 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:42.404 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:40:42.404 DEBUG:teuthology.orchestra.run.trial066:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T08:40:42.423 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T08:40:42.423 DEBUG:teuthology.orchestra.run.trial066:> stat /dev/vg_nvme/lv_1 2026-02-25T08:40:42.479 INFO:teuthology.orchestra.run.trial066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T08:40:42.479 INFO:teuthology.orchestra.run.trial066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:42.480 INFO:teuthology.orchestra.run.trial066.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-25T08:40:42.480 INFO:teuthology.orchestra.run.trial066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:42.480 INFO:teuthology.orchestra.run.trial066.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:42.480 INFO:teuthology.orchestra.run.trial066.stdout:Access: 2026-02-25 08:20:32.020497059 +0000 2026-02-25T08:40:42.480 INFO:teuthology.orchestra.run.trial066.stdout:Modify: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:42.480 INFO:teuthology.orchestra.run.trial066.stdout:Change: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:42.480 INFO:teuthology.orchestra.run.trial066.stdout: Birth: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:42.480 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T08:40:42.542 INFO:teuthology.orchestra.run.trial066.stderr:1+0 records in 2026-02-25T08:40:42.542 INFO:teuthology.orchestra.run.trial066.stderr:1+0 records out 2026-02-25T08:40:42.543 INFO:teuthology.orchestra.run.trial066.stderr:512 bytes copied, 0.000167904 s, 3.0 MB/s 2026-02-25T08:40:42.544 DEBUG:teuthology.orchestra.run.trial066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T08:40:42.600 DEBUG:teuthology.orchestra.run.trial066:> stat /dev/vg_nvme/lv_2 2026-02-25T08:40:42.657 INFO:teuthology.orchestra.run.trial066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T08:40:42.657 INFO:teuthology.orchestra.run.trial066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:42.657 INFO:teuthology.orchestra.run.trial066.stdout:Device: 6h/6d Inode: 1295 Links: 1 2026-02-25T08:40:42.657 INFO:teuthology.orchestra.run.trial066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:42.658 INFO:teuthology.orchestra.run.trial066.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:42.658 INFO:teuthology.orchestra.run.trial066.stdout:Access: 2026-02-25 08:20:32.020497059 +0000 2026-02-25T08:40:42.658 INFO:teuthology.orchestra.run.trial066.stdout:Modify: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:42.658 INFO:teuthology.orchestra.run.trial066.stdout:Change: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:42.658 INFO:teuthology.orchestra.run.trial066.stdout: Birth: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:42.658 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T08:40:42.720 INFO:teuthology.orchestra.run.trial066.stderr:1+0 records in 2026-02-25T08:40:42.720 INFO:teuthology.orchestra.run.trial066.stderr:1+0 records out 2026-02-25T08:40:42.720 INFO:teuthology.orchestra.run.trial066.stderr:512 bytes copied, 0.000163636 s, 3.1 MB/s 2026-02-25T08:40:42.721 DEBUG:teuthology.orchestra.run.trial066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T08:40:42.778 DEBUG:teuthology.orchestra.run.trial066:> stat /dev/vg_nvme/lv_3 2026-02-25T08:40:42.834 INFO:teuthology.orchestra.run.trial066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T08:40:42.835 INFO:teuthology.orchestra.run.trial066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:42.835 INFO:teuthology.orchestra.run.trial066.stdout:Device: 6h/6d Inode: 1316 Links: 1 2026-02-25T08:40:42.835 INFO:teuthology.orchestra.run.trial066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:42.835 INFO:teuthology.orchestra.run.trial066.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:42.835 INFO:teuthology.orchestra.run.trial066.stdout:Access: 2026-02-25 08:20:32.021497040 +0000 2026-02-25T08:40:42.835 INFO:teuthology.orchestra.run.trial066.stdout:Modify: 2026-02-25 08:09:45.735927710 +0000 2026-02-25T08:40:42.835 INFO:teuthology.orchestra.run.trial066.stdout:Change: 2026-02-25 08:09:45.735927710 +0000 2026-02-25T08:40:42.835 INFO:teuthology.orchestra.run.trial066.stdout: Birth: 2026-02-25 08:09:45.735927710 +0000 2026-02-25T08:40:42.836 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T08:40:42.897 INFO:teuthology.orchestra.run.trial066.stderr:1+0 records in 2026-02-25T08:40:42.897 INFO:teuthology.orchestra.run.trial066.stderr:1+0 records out 2026-02-25T08:40:42.897 INFO:teuthology.orchestra.run.trial066.stderr:512 bytes copied, 0.000154618 s, 3.3 MB/s 2026-02-25T08:40:42.898 DEBUG:teuthology.orchestra.run.trial066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T08:40:42.956 DEBUG:teuthology.orchestra.run.trial066:> stat /dev/vg_nvme/lv_4 2026-02-25T08:40:43.012 INFO:teuthology.orchestra.run.trial066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T08:40:43.012 INFO:teuthology.orchestra.run.trial066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:43.012 INFO:teuthology.orchestra.run.trial066.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-25T08:40:43.012 INFO:teuthology.orchestra.run.trial066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:43.012 INFO:teuthology.orchestra.run.trial066.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:43.012 INFO:teuthology.orchestra.run.trial066.stdout:Access: 2026-02-25 08:20:32.021497040 +0000 2026-02-25T08:40:43.012 INFO:teuthology.orchestra.run.trial066.stdout:Modify: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:43.013 INFO:teuthology.orchestra.run.trial066.stdout:Change: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:43.013 INFO:teuthology.orchestra.run.trial066.stdout: Birth: 2026-02-25 08:09:45.734927749 +0000 2026-02-25T08:40:43.013 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T08:40:43.074 INFO:teuthology.orchestra.run.trial066.stderr:1+0 records in 2026-02-25T08:40:43.075 INFO:teuthology.orchestra.run.trial066.stderr:1+0 records out 2026-02-25T08:40:43.075 INFO:teuthology.orchestra.run.trial066.stderr:512 bytes copied, 0.000130654 s, 3.9 MB/s 2026-02-25T08:40:43.076 DEBUG:teuthology.orchestra.run.trial066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T08:40:43.132 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:43.132 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial066.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-25T08:40:43.133 INFO:tasks.ceph:Generating config... 2026-02-25T08:40:43.135 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-25T08:40:43.135 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-25T08:40:43.135 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-25T08:40:43.136 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] osd delete sleep = 1 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] osd max backfills = 3 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2026-02-25T08:40:43.137 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2026-02-25T08:40:43.138 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-25T08:40:43.138 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2026-02-25T08:40:43.138 INFO:tasks.ceph:[osd] osd target transaction size = 5 2026-02-25T08:40:43.138 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-25T08:40:43.138 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-25T08:40:43.138 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-25T08:40:43.138 INFO:tasks.ceph:[client.0] admin socket = /var/run/ceph/ceph-$name.asok 2026-02-25T08:40:43.138 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-25T08:40:43.138 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-25T08:40:43.138 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-25T08:40:43.138 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-25T08:40:43.138 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-25T08:40:43.139 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-25T08:40:43.139 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-25T08:40:43.139 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2026-02-25T08:40:43.139 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2026-02-25T08:40:43.139 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-25T08:40:43.139 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-25T08:40:43.139 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-25T08:40:43.139 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-25T08:40:43.139 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-25T08:40:43.139 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-25T08:40:43.139 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-25T08:40:43.139 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-25T08:40:43.140 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-25T08:40:43.140 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-25T08:40:43.140 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-25T08:40:43.140 INFO:tasks.ceph:Setting up mon.a... 2026-02-25T08:40:43.140 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-25T08:40:43.183 INFO:teuthology.orchestra.run.trial042.stdout:creating /etc/ceph/ceph.keyring 2026-02-25T08:40:43.186 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-25T08:40:43.270 DEBUG:teuthology.orchestra.run.trial042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T08:40:43.334 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.42')] 2026-02-25T08:40:43.335 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.42', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}} 2026-02-25T08:40:43.337 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:43.337 DEBUG:teuthology.orchestra.run.trial042:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T08:40:43.391 DEBUG:teuthology.orchestra.run.trial042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.42 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:43.471 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:40:43.472+0000 7f8f9ac67f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.471 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:40:43.472+0000 7f8f9ac67f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.471 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:40:43.472+0000 7f8f9ac67f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:43.472 INFO:teuthology.orchestra.run.trial042.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:43.472 INFO:teuthology.orchestra.run.trial042.stdout:monmaptool: generated fsid 517215f2-d42c-4e04-9968-4190617e61fc 2026-02-25T08:40:43.472 INFO:teuthology.orchestra.run.trial042.stdout:setting min_mon_release = tentacle 2026-02-25T08:40:43.473 INFO:teuthology.orchestra.run.trial042.stdout:epoch 0 2026-02-25T08:40:43.473 INFO:teuthology.orchestra.run.trial042.stdout:fsid 517215f2-d42c-4e04-9968-4190617e61fc 2026-02-25T08:40:43.473 INFO:teuthology.orchestra.run.trial042.stdout:last_changed 2026-02-25T08:40:43.474354+0000 2026-02-25T08:40:43.473 INFO:teuthology.orchestra.run.trial042.stdout:created 2026-02-25T08:40:43.474354+0000 2026-02-25T08:40:43.473 INFO:teuthology.orchestra.run.trial042.stdout:min_mon_release 20 (tentacle) 2026-02-25T08:40:43.473 INFO:teuthology.orchestra.run.trial042.stdout:election_strategy: 1 2026-02-25T08:40:43.473 INFO:teuthology.orchestra.run.trial042.stdout:0: [v2:10.20.193.42:3300/0,v1:10.20.193.42:6789/0] mon.a 2026-02-25T08:40:43.473 INFO:teuthology.orchestra.run.trial042.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-25T08:40:43.474 DEBUG:teuthology.orchestra.run.trial042:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T08:40:43.529 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 517215f2-d42c-4e04-9968-4190617e61fc... 2026-02-25T08:40:43.530 DEBUG:teuthology.orchestra.run.trial042:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T08:40:43.571 DEBUG:teuthology.orchestra.run.trial066:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T08:40:43.610 INFO:teuthology.orchestra.run.trial042.stdout:[global] 2026-02-25T08:40:43.610 INFO:teuthology.orchestra.run.trial042.stdout: chdir = "" 2026-02-25T08:40:43.610 INFO:teuthology.orchestra.run.trial042.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T08:40:43.610 INFO:teuthology.orchestra.run.trial042.stdout: auth supported = cephx 2026-02-25T08:40:43.610 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.610 INFO:teuthology.orchestra.run.trial042.stdout: filestore xattr use omap = true 2026-02-25T08:40:43.610 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.610 INFO:teuthology.orchestra.run.trial042.stdout: mon clock drift allowed = 1.000 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: osd crush chooseleaf type = 0 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: auth debug = true 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: ms die on old message = true 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: ms die on bug = true 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: mon pg warn max object skew = 0 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T08:40:43.611 INFO:teuthology.orchestra.run.trial042.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: osd pool default size = 2 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon osd allow primary affinity = true 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon osd allow pg remap = true 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon warn on legacy crush tunables = false 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon warn on crush straw calc version zero = false 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon warn on no sortbitwise = false 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon warn on osd down out interval zero = false 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon warn on too few osds = false 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T08:40:43.612 INFO:teuthology.orchestra.run.trial042.stdout: mon_allow_pool_size_one = true 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: osd default data pool replay window = 5 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: mon allow pool delete = true 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: mon cluster log file level = debug 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: debug asserts on shutdown = true 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: mon health detail to clog = false 2026-02-25T08:40:43.613 INFO:teuthology.orchestra.run.trial042.stdout: mon host = 10.20.193.42 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: ms client mode = crc 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: ms cluster mode = crc 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: ms mon client mode = crc 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: ms mon cluster mode = crc 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: ms mon service mode = crc 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: ms service mode = crc 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: osd_pool_default_min_size = 2 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: osd_pool_default_size = 3 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: fsid = 517215f2-d42c-4e04-9968-4190617e61fc 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.614 INFO:teuthology.orchestra.run.trial042.stdout:[osd] 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd journal size = 100 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd scrub load threshold = 5.0 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd scrub max interval = 120 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd mclock profile = high_recovery_ops 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd mclock skip benchmark = true 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd recover clone overlap = true 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd recovery max chunk = 1048576 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd debug shutdown = true 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd debug op order = true 2026-02-25T08:40:43.615 INFO:teuthology.orchestra.run.trial042.stdout: osd debug verify stray on activate = true 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: osd debug trim objects = true 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: osd open classes on start = true 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: osd debug pg log writeout = true 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: osd deep scrub update digest min age = 30 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: osd map max advance = 10 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: journal zero on create = true 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: filestore ondisk finisher threads = 3 2026-02-25T08:40:43.616 INFO:teuthology.orchestra.run.trial042.stdout: filestore apply finisher threads = 3 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: bdev debug aio = true 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: osd debug misdirected ops = true 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: debug monc = 20 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: crimson cpu num = 4 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: crimson osd obc lru size = 10 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: debug ms = 20 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: debug osd = 20 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore = 20 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore backref = 20 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore cache = 20 2026-02-25T08:40:43.617 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore cleaner = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore device = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore epm = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore fixedkv tree = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore journal = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore lba = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore odata = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore omap = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore onode = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore t = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: debug seastore tm = 20 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: osd blocked scrub grace period = 3600 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: osd debug reject backfill probability = 0.3 2026-02-25T08:40:43.618 INFO:teuthology.orchestra.run.trial042.stdout: osd delete sleep = 1 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: osd max backfills = 3 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: osd max markdown count = 1000 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: osd objectstore = seastore 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: osd scrub min interval = 60 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: osd shutdown pgref assert = True 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: osd snap trim sleep = 2 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: osd target transaction size = 5 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: seastore cachepin size pershard = 64M 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: seastore max concurrent transactions = 8 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.619 INFO:teuthology.orchestra.run.trial042.stdout:[mgr] 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: debug ms = 1 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: debug mgr = 20 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: debug mon = 20 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: debug auth = 20 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: mgr/telemetry/nag = false 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout:[mon] 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: debug ms = 1 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: debug mon = 20 2026-02-25T08:40:43.620 INFO:teuthology.orchestra.run.trial042.stdout: debug paxos = 20 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: debug auth = 20 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: mon data avail warn = 5 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: mon mgr mkfs grace = 240 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: mon osd reporter subtree level = osd 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: mon osd prime pg temp = true 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T08:40:43.621 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: # 1m isn't quite enough 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: mon_down_mkfs_grace = 2m 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: mon_warn_on_filestore_osds = false 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: mon min osdmap epochs = 50 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: mon osdmap full prune interval = 2 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: mon osdmap full prune min = 15 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: mon osdmap full prune txsize = 2 2026-02-25T08:40:43.622 INFO:teuthology.orchestra.run.trial042.stdout: osd pool default crimson = True 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout: paxos service trim min = 10 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout:[client] 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout: rgw cache enabled = true 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout: rgw enable ops log = true 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout: rgw enable usage log = true 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout:[mon.a] 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout:[client.0] 2026-02-25T08:40:43.623 INFO:teuthology.orchestra.run.trial042.stdout: admin socket = /var/run/ceph/ceph-$name.asok 2026-02-25T08:40:43.625 INFO:teuthology.orchestra.run.trial066.stdout:[global] 2026-02-25T08:40:43.625 INFO:teuthology.orchestra.run.trial066.stdout: chdir = "" 2026-02-25T08:40:43.625 INFO:teuthology.orchestra.run.trial066.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T08:40:43.625 INFO:teuthology.orchestra.run.trial066.stdout: auth supported = cephx 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: filestore xattr use omap = true 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: mon clock drift allowed = 1.000 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: osd crush chooseleaf type = 0 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: auth debug = true 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: ms die on old message = true 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: ms die on bug = true 2026-02-25T08:40:43.626 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: mon pg warn max object skew = 0 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: osd pool default size = 2 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: mon osd allow primary affinity = true 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: mon osd allow pg remap = true 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: mon warn on legacy crush tunables = false 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: mon warn on crush straw calc version zero = false 2026-02-25T08:40:43.627 INFO:teuthology.orchestra.run.trial066.stdout: mon warn on no sortbitwise = false 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: mon warn on osd down out interval zero = false 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: mon warn on too few osds = false 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: mon_allow_pool_size_one = true 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: osd default data pool replay window = 5 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: mon allow pool delete = true 2026-02-25T08:40:43.628 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: mon cluster log file level = debug 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: debug asserts on shutdown = true 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: mon health detail to clog = false 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: mon host = 10.20.193.42 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: ms client mode = crc 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: ms cluster mode = crc 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: ms mon client mode = crc 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: ms mon cluster mode = crc 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: ms mon service mode = crc 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: ms service mode = crc 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: osd_pool_default_min_size = 2 2026-02-25T08:40:43.629 INFO:teuthology.orchestra.run.trial066.stdout: osd_pool_default_size = 3 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: fsid = 517215f2-d42c-4e04-9968-4190617e61fc 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout:[osd] 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: osd journal size = 100 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: osd scrub load threshold = 5.0 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: osd scrub max interval = 120 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: osd mclock profile = high_recovery_ops 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: osd mclock skip benchmark = true 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: osd recover clone overlap = true 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: osd recovery max chunk = 1048576 2026-02-25T08:40:43.630 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: osd debug shutdown = true 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: osd debug op order = true 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: osd debug verify stray on activate = true 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: osd debug trim objects = true 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: osd open classes on start = true 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: osd debug pg log writeout = true 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: osd deep scrub update digest min age = 30 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: osd map max advance = 10 2026-02-25T08:40:43.631 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: journal zero on create = true 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: filestore ondisk finisher threads = 3 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: filestore apply finisher threads = 3 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: bdev debug aio = true 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: osd debug misdirected ops = true 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: debug monc = 20 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: crimson cpu num = 4 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: crimson osd obc lru size = 10 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: debug ms = 20 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: debug osd = 20 2026-02-25T08:40:43.632 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore backref = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore cache = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore cleaner = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore device = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore epm = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore fixedkv tree = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore journal = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore lba = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore odata = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore omap = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore onode = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore t = 20 2026-02-25T08:40:43.633 INFO:teuthology.orchestra.run.trial066.stdout: debug seastore tm = 20 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd blocked scrub grace period = 3600 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd debug reject backfill probability = 0.3 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd delete sleep = 1 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd max backfills = 3 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd max markdown count = 1000 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd objectstore = seastore 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd scrub min interval = 60 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd shutdown pgref assert = True 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd snap trim sleep = 2 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: osd target transaction size = 5 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: seastore cachepin size pershard = 64M 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: seastore max concurrent transactions = 8 2026-02-25T08:40:43.634 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout:[mgr] 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: debug ms = 1 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: debug mgr = 20 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: debug mon = 20 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: debug auth = 20 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: mgr/telemetry/nag = false 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout:[mon] 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: debug ms = 1 2026-02-25T08:40:43.635 INFO:teuthology.orchestra.run.trial066.stdout: debug mon = 20 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: debug paxos = 20 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: debug auth = 20 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: mon data avail warn = 5 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: mon mgr mkfs grace = 240 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: mon osd reporter subtree level = osd 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: mon osd prime pg temp = true 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T08:40:43.636 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: # 1m isn't quite enough 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: mon_down_mkfs_grace = 2m 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: mon_warn_on_filestore_osds = false 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: mon min osdmap epochs = 50 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: mon osdmap full prune interval = 2 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: mon osdmap full prune min = 15 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: mon osdmap full prune txsize = 2 2026-02-25T08:40:43.637 INFO:teuthology.orchestra.run.trial066.stdout: osd pool default crimson = True 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout: paxos service trim min = 10 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout: 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout:[client] 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout: rgw cache enabled = true 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout: rgw enable ops log = true 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout: rgw enable usage log = true 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout:[mon.a] 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout:[client.0] 2026-02-25T08:40:43.638 INFO:teuthology.orchestra.run.trial066.stdout: admin socket = /var/run/ceph/ceph-$name.asok 2026-02-25T08:40:43.639 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-25T08:40:43.639 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-25T08:40:43.716 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-25T08:40:43.716 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:43.716 DEBUG:teuthology.orchestra.run.trial042:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-25T08:40:43.732 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:43.733 DEBUG:teuthology.orchestra.run.trial042:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-25T08:40:43.789 INFO:tasks.ceph:Sending monmap to node ubuntu@trial042.front.sepia.ceph.com 2026-02-25T08:40:43.789 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:43.789 DEBUG:teuthology.orchestra.run.trial042:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T08:40:43.789 DEBUG:teuthology.orchestra.run.trial042:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T08:40:43.861 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:40:43.861 DEBUG:teuthology.orchestra.run.trial042:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:43.917 INFO:tasks.ceph:Sending monmap to node ubuntu@trial066.front.sepia.ceph.com 2026-02-25T08:40:43.917 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:40:43.917 DEBUG:teuthology.orchestra.run.trial066:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T08:40:43.917 DEBUG:teuthology.orchestra.run.trial066:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T08:40:43.951 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:40:43.951 DEBUG:teuthology.orchestra.run.trial066:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:44.008 INFO:tasks.ceph:Setting up mon nodes... 2026-02-25T08:40:44.008 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-25T08:40:44.008 DEBUG:teuthology.orchestra.run.trial066:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T08:40:44.099 INFO:teuthology.orchestra.run.trial066.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T08:40:44.102 INFO:tasks.ceph:Setting up mds nodes... 2026-02-25T08:40:44.102 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-25T08:40:44.103 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-25T08:40:44.146 INFO:teuthology.orchestra.run.trial042.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-25T08:40:44.158 DEBUG:teuthology.orchestra.run.trial066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-25T08:40:44.202 INFO:teuthology.orchestra.run.trial066.stdout:creating /etc/ceph/ceph.client.1.keyring 2026-02-25T08:40:44.213 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-25T08:40:44.214 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial042.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial066.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-25T08:40:44.214 DEBUG:teuthology.orchestra.run.trial042:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:44.239 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:44.239 INFO:tasks.ceph:role: osd.0 2026-02-25T08:40:44.239 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial042.front.sepia.ceph.com 2026-02-25T08:40:44.239 DEBUG:teuthology.orchestra.run.trial042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T08:40:44.303 INFO:teuthology.orchestra.run.trial042.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:44.303 INFO:teuthology.orchestra.run.trial042.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:44.303 INFO:teuthology.orchestra.run.trial042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:44.303 INFO:teuthology.orchestra.run.trial042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:44.303 INFO:teuthology.orchestra.run.trial042.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:44.303 INFO:teuthology.orchestra.run.trial042.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:44.304 INFO:teuthology.orchestra.run.trial042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:44.304 INFO:teuthology.orchestra.run.trial042.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:44.304 INFO:teuthology.orchestra.run.trial042.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:44.304 INFO:teuthology.orchestra.run.trial042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:44.884 INFO:teuthology.orchestra.run.trial042.stdout:Discarding blocks...Done. 2026-02-25T08:40:44.887 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial042.front.sepia.ceph.com -o noatime 2026-02-25T08:40:44.887 DEBUG:teuthology.orchestra.run.trial042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:44.956 DEBUG:teuthology.orchestra.run.trial042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:45.023 DEBUG:teuthology.orchestra.run.trial042:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:45.087 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:45.087 INFO:tasks.ceph:role: osd.1 2026-02-25T08:40:45.087 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial042.front.sepia.ceph.com 2026-02-25T08:40:45.087 DEBUG:teuthology.orchestra.run.trial042:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T08:40:45.152 INFO:teuthology.orchestra.run.trial042.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:45.152 INFO:teuthology.orchestra.run.trial042.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:45.152 INFO:teuthology.orchestra.run.trial042.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:45.152 INFO:teuthology.orchestra.run.trial042.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:45.152 INFO:teuthology.orchestra.run.trial042.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:45.152 INFO:teuthology.orchestra.run.trial042.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:45.152 INFO:teuthology.orchestra.run.trial042.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:45.152 INFO:teuthology.orchestra.run.trial042.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:45.153 INFO:teuthology.orchestra.run.trial042.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:45.153 INFO:teuthology.orchestra.run.trial042.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:45.720 INFO:teuthology.orchestra.run.trial042.stdout:Discarding blocks...Done. 2026-02-25T08:40:45.722 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial042.front.sepia.ceph.com -o noatime 2026-02-25T08:40:45.722 DEBUG:teuthology.orchestra.run.trial042:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:45.791 DEBUG:teuthology.orchestra.run.trial042:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:45.857 DEBUG:teuthology.orchestra.run.trial042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:46.006 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:46.063 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:46,067 seastar - Reactor backend: linux-aio 2026-02-25T08:40:46.076 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:46,080 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:46.077 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:46,080 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:46.079 INFO:teuthology.orchestra.run.trial042.stderr:==39525==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:46.135 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:46,138 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T08:40:46.135 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:46,138 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T08:40:46.135 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:46,138 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T08:40:46.135 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:46,138 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T08:40:46.136 INFO:teuthology.orchestra.run.trial042.stderr:DEBUG 2026-02-25 08:40:46,138 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:48.201 INFO:teuthology.orchestra.run.trial042.stderr:----------------------------------------------------- 2026-02-25T08:40:48.202 INFO:teuthology.orchestra.run.trial042.stderr:Suppressions used: 2026-02-25T08:40:48.202 INFO:teuthology.orchestra.run.trial042.stderr: count bytes template 2026-02-25T08:40:48.202 INFO:teuthology.orchestra.run.trial042.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:48.202 INFO:teuthology.orchestra.run.trial042.stderr:----------------------------------------------------- 2026-02-25T08:40:48.202 INFO:teuthology.orchestra.run.trial042.stderr: 2026-02-25T08:40:48.220 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:48.221 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:48.269 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:48.269 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:48.275 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:48,279 seastar - Reactor backend: linux-aio 2026-02-25T08:40:48.286 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:48,289 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:48.286 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:48,289 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:48.290 INFO:teuthology.orchestra.run.trial042.stderr:==39523==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:48.290 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:48,293 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:48.348 INFO:teuthology.orchestra.run.trial042.stderr:DEBUG 2026-02-25 08:40:48,352 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T08:40:48.428 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe2f10b 0xe2f341 0x7ba1708 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:48.428 INFO:teuthology.orchestra.run.trial042.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa6f636e6 0xffffffffa6f634a5 0xffffffffa65d19d8 0xffffffffa65d4bab 0xffffffffa65d500b 0xffffffffa62910d2 0xffffffffa6f50451 0xffffffffa7000ba2 2026-02-25T08:40:48.429 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe2f10b 0xe2f42e 0x9ef6eb8 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:48.429 INFO:teuthology.orchestra.run.trial042.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa65ca9cd 0xffffffffa65d4bab 0xffffffffa65d500b 0xffffffffa62910d2 0xffffffffa6f50451 0xffffffffa7000ba2 2026-02-25T08:40:48.429 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0xbb1391 0xc1c274 0xc1c309 0xc85d77 0xa7a7e69 0x9f193cc 0x9f195b3 0x9f1977c 0x9f1980f 0x9f1a654 0x78b69db 0x7ba11e3 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:48.580 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe40f5d 0xe412b4 0x74b30c8 0x74b31c9 0x731ea62 0x78b693f 0x7ba11e3 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:48.580 INFO:teuthology.orchestra.run.trial042.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa641fee1 0xffffffffa64203b0 0xffffffffa629525e 0xffffffffa65e7269 0xffffffffa65ca7d5 0xffffffffa65d4bab 0xffffffffa65d500b 0xffffffffa62910d2 0xffffffffa6f50451 0xffffffffa7000ba2 2026-02-25T08:40:48.580 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c62 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe40f5d 0x9d7520f 0x9eec474 0x9eec56d 0x7ba1825 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:48.580 INFO:teuthology.orchestra.run.trial042.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa6f636e6 0xffffffffa6f634a5 0xffffffffa65e7985 0xffffffffa65cc97a 0xffffffffa65d4bab 0xffffffffa65d500b 0xffffffffa62910d2 0xffffffffa6f50451 0xffffffffa7000ba2 2026-02-25T08:40:48.580 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xb7ac87 0x10a27ac 0xf47938 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:50.736 INFO:teuthology.orchestra.run.trial042.stderr:----------------------------------------------------- 2026-02-25T08:40:50.736 INFO:teuthology.orchestra.run.trial042.stderr:Suppressions used: 2026-02-25T08:40:50.736 INFO:teuthology.orchestra.run.trial042.stderr: count bytes template 2026-02-25T08:40:50.736 INFO:teuthology.orchestra.run.trial042.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:50.736 INFO:teuthology.orchestra.run.trial042.stderr:----------------------------------------------------- 2026-02-25T08:40:50.737 INFO:teuthology.orchestra.run.trial042.stderr: 2026-02-25T08:40:50.767 DEBUG:teuthology.orchestra.run.trial042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:50.792 DEBUG:teuthology.orchestra.run.trial042:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:50.942 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:50.999 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:51,002 seastar - Reactor backend: linux-aio 2026-02-25T08:40:51.008 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:51,012 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:51.008 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:51,012 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:51.010 INFO:teuthology.orchestra.run.trial042.stderr:==39593==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:51.067 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:51,070 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T08:40:51.067 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:51,070 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T08:40:51.067 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:51,070 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T08:40:51.067 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:51,070 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T08:40:51.068 INFO:teuthology.orchestra.run.trial042.stderr:DEBUG 2026-02-25 08:40:51,070 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:53.129 INFO:teuthology.orchestra.run.trial042.stderr:----------------------------------------------------- 2026-02-25T08:40:53.130 INFO:teuthology.orchestra.run.trial042.stderr:Suppressions used: 2026-02-25T08:40:53.130 INFO:teuthology.orchestra.run.trial042.stderr: count bytes template 2026-02-25T08:40:53.130 INFO:teuthology.orchestra.run.trial042.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:53.130 INFO:teuthology.orchestra.run.trial042.stderr:----------------------------------------------------- 2026-02-25T08:40:53.130 INFO:teuthology.orchestra.run.trial042.stderr: 2026-02-25T08:40:53.149 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:53.149 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:53.190 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:53.190 INFO:teuthology.orchestra.run.trial042.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:53.197 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:53,200 seastar - Reactor backend: linux-aio 2026-02-25T08:40:53.211 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:53,215 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:53.211 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:53,215 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:53.215 INFO:teuthology.orchestra.run.trial042.stderr:==39591==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:53.215 INFO:teuthology.orchestra.run.trial042.stderr:INFO 2026-02-25 08:40:53,219 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:53.277 INFO:teuthology.orchestra.run.trial042.stderr:DEBUG 2026-02-25 08:40:53,281 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T08:40:53.356 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xea9cf 0xe40c90 0xe4104b 0x9f16c08 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.357 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x7ba1249 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.357 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x10a26bf 0xf47938 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9e24090 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.515 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xed74e 0xcc26c1 0x6f43408 0x9ef901e 0x9ef9236 0x9ef9508 0x7ba18ef 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.515 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xcc2811 0x10a28e0 0xf47938 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9f0751a 0x9f0916c 0x9f097c6 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.515 INFO:teuthology.orchestra.run.trial042.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x351b46c 0x35fc112 0x35fc215 0x3618092 0x9e33da7 0x9e33e69 0x9ef3c57 0x9ef3dec 0x9ef417e 0x9ef5765 0x9ef6a80 0x9ef6ea0 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:55.667 INFO:teuthology.orchestra.run.trial042.stderr:----------------------------------------------------- 2026-02-25T08:40:55.668 INFO:teuthology.orchestra.run.trial042.stderr:Suppressions used: 2026-02-25T08:40:55.668 INFO:teuthology.orchestra.run.trial042.stderr: count bytes template 2026-02-25T08:40:55.668 INFO:teuthology.orchestra.run.trial042.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:55.668 INFO:teuthology.orchestra.run.trial042.stderr:----------------------------------------------------- 2026-02-25T08:40:55.668 INFO:teuthology.orchestra.run.trial042.stderr: 2026-02-25T08:40:55.698 DEBUG:teuthology.orchestra.run.trial042:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:55.724 DEBUG:teuthology.orchestra.run.trial066:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:55.751 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:55.751 INFO:tasks.ceph:role: osd.2 2026-02-25T08:40:55.751 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial066.front.sepia.ceph.com 2026-02-25T08:40:55.751 DEBUG:teuthology.orchestra.run.trial066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T08:40:55.815 INFO:teuthology.orchestra.run.trial066.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:55.815 INFO:teuthology.orchestra.run.trial066.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:55.816 INFO:teuthology.orchestra.run.trial066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:55.816 INFO:teuthology.orchestra.run.trial066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:55.816 INFO:teuthology.orchestra.run.trial066.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:55.816 INFO:teuthology.orchestra.run.trial066.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:55.816 INFO:teuthology.orchestra.run.trial066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:55.816 INFO:teuthology.orchestra.run.trial066.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:55.816 INFO:teuthology.orchestra.run.trial066.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:55.816 INFO:teuthology.orchestra.run.trial066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:56.385 INFO:teuthology.orchestra.run.trial066.stdout:Discarding blocks...Done. 2026-02-25T08:40:56.387 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial066.front.sepia.ceph.com -o noatime 2026-02-25T08:40:56.388 DEBUG:teuthology.orchestra.run.trial066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:56.457 DEBUG:teuthology.orchestra.run.trial066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:56.524 DEBUG:teuthology.orchestra.run.trial066:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-25T08:40:56.588 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T08:40:56.588 INFO:tasks.ceph:role: osd.3 2026-02-25T08:40:56.588 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial066.front.sepia.ceph.com 2026-02-25T08:40:56.588 DEBUG:teuthology.orchestra.run.trial066:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T08:40:56.653 INFO:teuthology.orchestra.run.trial066.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:56.653 INFO:teuthology.orchestra.run.trial066.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:56.653 INFO:teuthology.orchestra.run.trial066.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:56.653 INFO:teuthology.orchestra.run.trial066.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:56.653 INFO:teuthology.orchestra.run.trial066.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:56.653 INFO:teuthology.orchestra.run.trial066.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:56.653 INFO:teuthology.orchestra.run.trial066.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:56.654 INFO:teuthology.orchestra.run.trial066.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:56.654 INFO:teuthology.orchestra.run.trial066.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:56.654 INFO:teuthology.orchestra.run.trial066.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:57.212 INFO:teuthology.orchestra.run.trial066.stdout:Discarding blocks...Done. 2026-02-25T08:40:57.214 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial066.front.sepia.ceph.com -o noatime 2026-02-25T08:40:57.214 DEBUG:teuthology.orchestra.run.trial066:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2026-02-25T08:40:57.282 DEBUG:teuthology.orchestra.run.trial066:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-25T08:40:57.348 DEBUG:teuthology.orchestra.run.trial066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:57.499 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:57.556 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:57,555 seastar - Reactor backend: linux-aio 2026-02-25T08:40:57.565 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:57,565 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:57.565 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:57,565 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:57.567 INFO:teuthology.orchestra.run.trial066.stderr:==39437==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:57.623 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:57,622 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T08:40:57.623 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:57,622 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T08:40:57.624 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:57,622 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T08:40:57.624 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:57,622 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T08:40:57.624 INFO:teuthology.orchestra.run.trial066.stderr:DEBUG 2026-02-25 08:40:57,623 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:59.667 INFO:teuthology.orchestra.run.trial066.stderr:----------------------------------------------------- 2026-02-25T08:40:59.667 INFO:teuthology.orchestra.run.trial066.stderr:Suppressions used: 2026-02-25T08:40:59.667 INFO:teuthology.orchestra.run.trial066.stderr: count bytes template 2026-02-25T08:40:59.667 INFO:teuthology.orchestra.run.trial066.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:59.667 INFO:teuthology.orchestra.run.trial066.stderr:----------------------------------------------------- 2026-02-25T08:40:59.668 INFO:teuthology.orchestra.run.trial066.stderr: 2026-02-25T08:40:59.686 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:59.686 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:59.727 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:59.727 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:59.734 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:59,734 seastar - Reactor backend: linux-aio 2026-02-25T08:40:59.745 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:59,745 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:59.746 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:59,745 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:59.749 INFO:teuthology.orchestra.run.trial066.stderr:==39435==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:59.750 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:40:59,749 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:59.808 INFO:teuthology.orchestra.run.trial066.stderr:DEBUG 2026-02-25 08:40:59,808 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T08:40:59.888 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x10a26f2 0xf47938 0x9f044e9 0x9f04ad6 0x9f04e59 0x9f14fb0 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:59.888 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129787 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22694 0x9e2ab3b 0x9e2b21b 0x9f13d68 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:59.888 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x9e21748 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9e24090 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:00.038 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x9e323cd 0x9f0f898 0x9f0fe6a 0x9f1a337 0x78b69db 0x7ba11e3 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:00.038 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x157e6 0xca7484 0xcb95ed 0x9e695a5 0x9eeaeb6 0x9eeb09e 0x9eeb14a 0x9f13269 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:00.038 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65d9 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9e24090 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:02.182 INFO:teuthology.orchestra.run.trial066.stderr:----------------------------------------------------- 2026-02-25T08:41:02.182 INFO:teuthology.orchestra.run.trial066.stderr:Suppressions used: 2026-02-25T08:41:02.182 INFO:teuthology.orchestra.run.trial066.stderr: count bytes template 2026-02-25T08:41:02.182 INFO:teuthology.orchestra.run.trial066.stderr: 1 8 MallocExtension::Register 2026-02-25T08:41:02.182 INFO:teuthology.orchestra.run.trial066.stderr:----------------------------------------------------- 2026-02-25T08:41:02.182 INFO:teuthology.orchestra.run.trial066.stderr: 2026-02-25T08:41:02.215 DEBUG:teuthology.orchestra.run.trial066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-25T08:41:02.240 DEBUG:teuthology.orchestra.run.trial066:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:41:02.391 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:41:02.450 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:02,450 seastar - Reactor backend: linux-aio 2026-02-25T08:41:02.468 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:02,468 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:41:02.469 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:02,468 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:41:02.471 INFO:teuthology.orchestra.run.trial066.stderr:==39505==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:41:02.527 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:02,526 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T08:41:02.527 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:02,526 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T08:41:02.527 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:02,526 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T08:41:02.527 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:02,526 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T08:41:02.528 INFO:teuthology.orchestra.run.trial066.stderr:DEBUG 2026-02-25 08:41:02,527 [shard 0:main] seastar - reactor::drain 2026-02-25T08:41:04.586 INFO:teuthology.orchestra.run.trial066.stderr:----------------------------------------------------- 2026-02-25T08:41:04.586 INFO:teuthology.orchestra.run.trial066.stderr:Suppressions used: 2026-02-25T08:41:04.586 INFO:teuthology.orchestra.run.trial066.stderr: count bytes template 2026-02-25T08:41:04.586 INFO:teuthology.orchestra.run.trial066.stderr: 1 8 MallocExtension::Register 2026-02-25T08:41:04.586 INFO:teuthology.orchestra.run.trial066.stderr:----------------------------------------------------- 2026-02-25T08:41:04.587 INFO:teuthology.orchestra.run.trial066.stderr: 2026-02-25T08:41:04.606 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:41:04.606 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:41:04.654 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:41:04.655 INFO:teuthology.orchestra.run.trial066.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:41:04.661 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:04,660 seastar - Reactor backend: linux-aio 2026-02-25T08:41:04.672 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:04,672 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:41:04.673 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:04,672 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:41:04.676 INFO:teuthology.orchestra.run.trial066.stderr:==39503==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:41:04.676 INFO:teuthology.orchestra.run.trial066.stderr:INFO 2026-02-25 08:41:04,676 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:41:04.735 INFO:teuthology.orchestra.run.trial066.stderr:DEBUG 2026-02-25 08:41:04,734 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T08:41:04.815 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129787 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0xca7484 0xcb95ed 0x9e1cc62 0x9f1863a 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:04.815 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xcca042 0x113805f 0xf4792d 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:04.815 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16eaf /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x157c7 0x10a2823 0xf47938 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e22e1c 0x9e2338a 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:04.967 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x11380c8 0xf4792d 0x9e26106 0x9e263b3 0x9f138a5 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:04.967 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcaef /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9f077c6 0x9f0916c 0x9f097c6 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:04.967 INFO:teuthology.orchestra.run.trial066.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xcca009 0x113805f 0xf4792d 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9e24090 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:07.119 INFO:teuthology.orchestra.run.trial066.stderr:----------------------------------------------------- 2026-02-25T08:41:07.119 INFO:teuthology.orchestra.run.trial066.stderr:Suppressions used: 2026-02-25T08:41:07.120 INFO:teuthology.orchestra.run.trial066.stderr: count bytes template 2026-02-25T08:41:07.120 INFO:teuthology.orchestra.run.trial066.stderr: 1 8 MallocExtension::Register 2026-02-25T08:41:07.120 INFO:teuthology.orchestra.run.trial066.stderr:----------------------------------------------------- 2026-02-25T08:41:07.120 INFO:teuthology.orchestra.run.trial066.stderr: 2026-02-25T08:41:07.150 DEBUG:teuthology.orchestra.run.trial066:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-25T08:41:07.176 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-25T08:41:07.176 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:41:07.177 DEBUG:teuthology.orchestra.run.trial042:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-25T08:41:07.202 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:41:07.203 DEBUG:teuthology.orchestra.run.trial042:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-25T08:41:07.266 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:41:07.266 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-25T08:41:07.292 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:41:07.292 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-25T08:41:07.355 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:41:07.355 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-25T08:41:07.419 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:41:07.420 DEBUG:teuthology.orchestra.run.trial042:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-25T08:41:07.437 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:41:07.437 DEBUG:teuthology.orchestra.run.trial066:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2026-02-25T08:41:07.476 INFO:tasks.ceph:Adding keys to all mons... 2026-02-25T08:41:07.476 DEBUG:teuthology.orchestra.run.trial042:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-25T08:41:07.501 INFO:teuthology.orchestra.run.trial042.stdout:[osd.0] 2026-02-25T08:41:07.501 INFO:teuthology.orchestra.run.trial042.stdout: key = AQCQtZ5pLVE7FRAAqb1T9tMgY+16oSva4CzXXQ== 2026-02-25T08:41:07.501 INFO:teuthology.orchestra.run.trial042.stdout:[osd.1] 2026-02-25T08:41:07.501 INFO:teuthology.orchestra.run.trial042.stdout: key = AQCVtZ5p/2P5EBAA3xdYzSAERgHlh+SqVlY9mA== 2026-02-25T08:41:07.501 INFO:teuthology.orchestra.run.trial042.stdout:[mgr.x] 2026-02-25T08:41:07.501 INFO:teuthology.orchestra.run.trial042.stdout: key = AQCMtZ5pHHfqBRAAeCuB6hxPRuMEyfyQ78VYew== 2026-02-25T08:41:07.501 INFO:teuthology.orchestra.run.trial042.stdout:[osd.2] 2026-02-25T08:41:07.501 INFO:teuthology.orchestra.run.trial042.stdout: key = AQCbtZ5phr1pMBAAD3f/FcvA4vbRrRUPx0BdqQ== 2026-02-25T08:41:07.502 INFO:teuthology.orchestra.run.trial042.stdout:[osd.3] 2026-02-25T08:41:07.502 INFO:teuthology.orchestra.run.trial042.stdout: key = AQCgtZ5p0CEBLBAATyDhBbKHphn8pkjHU59XHA== 2026-02-25T08:41:07.502 INFO:teuthology.orchestra.run.trial042.stdout:[client.0] 2026-02-25T08:41:07.502 INFO:teuthology.orchestra.run.trial042.stdout: key = AQCMtZ5pZ4DpCBAAsmFH22EO20qe2POPJhRUzA== 2026-02-25T08:41:07.502 INFO:teuthology.orchestra.run.trial042.stdout:[client.1] 2026-02-25T08:41:07.502 INFO:teuthology.orchestra.run.trial042.stdout: key = AQCMtZ5pIFEJDBAAlEHDH6QsUpXJfv1XDGofQg== 2026-02-25T08:41:07.503 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:41:07.590 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:41:07.638 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T08:41:07.686 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:41:07.734 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:41:07.783 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T08:41:07.833 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T08:41:07.882 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-25T08:41:07.882 DEBUG:teuthology.orchestra.run.trial042:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-25T08:41:07.907 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-25T08:41:07.992 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:07.995+0000 7f051065ec00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.992 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:07.995+0000 7f051065ec00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.004 DEBUG:teuthology.orchestra.run.trial042:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-25T08:41:08.028 DEBUG:teuthology.orchestra.run.trial042:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:41:08.085 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-25T08:41:08.085 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-25T08:41:08.086 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-25T08:41:08.127 INFO:tasks.ceph.mon.a:Started 2026-02-25T08:41:08.127 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-25T08:41:08.128 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-25T08:41:08.128 DEBUG:teuthology.orchestra.run.trial066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-25T08:41:08.130 INFO:tasks.ceph.mgr.x:Started 2026-02-25T08:41:08.131 DEBUG:tasks.ceph:set 0 configs 2026-02-25T08:41:08.131 DEBUG:teuthology.orchestra.run.trial042:> sudo ceph --cluster ceph config dump 2026-02-25T08:41:08.187 INFO:tasks.ceph.mon.a.trial042.stderr:2026-02-25T08:41:08.190+0000 7f89d6c88c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.187 INFO:tasks.ceph.mon.a.trial042.stderr:2026-02-25T08:41:08.190+0000 7f89d6c88c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.187 INFO:tasks.ceph.mon.a.trial042.stderr:2026-02-25T08:41:08.190+0000 7f89d6c88c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.190 INFO:tasks.ceph.mgr.x.trial066.stderr:2026-02-25T08:41:08.189+0000 7f4ca0bc5fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.203 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:08.204+0000 7fb3565a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.209 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:08.212+0000 7fb3565a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.209 INFO:tasks.ceph.mgr.x.trial066.stderr:2026-02-25T08:41:08.208+0000 7f4ca0bc5fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.209 INFO:tasks.ceph.mgr.x.trial066.stderr:2026-02-25T08:41:08.209+0000 7f4ca0bc5fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.241 INFO:teuthology.orchestra.run.trial042.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-25T08:41:08.250 INFO:tasks.ceph:Setting crush tunables to default 2026-02-25T08:41:08.250 DEBUG:teuthology.orchestra.run.trial042:> sudo ceph --cluster ceph osd crush tunables default 2026-02-25T08:41:08.324 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:08.324+0000 7f5da9183640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.326 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:08.330+0000 7f5da9183640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.361 INFO:teuthology.orchestra.run.trial042.stderr:adjusted tunables profile to default 2026-02-25T08:41:08.369 INFO:tasks.ceph:check_enable_crimson: True 2026-02-25T08:41:08.370 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-25T08:41:08.370 DEBUG:teuthology.orchestra.run.trial042:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-25T08:41:08.442 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:08.442+0000 7f3049411640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.444 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:08.447+0000 7f3049411640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.483 INFO:tasks.ceph.mon.a.trial042.stderr:2026-02-25T08:41:08.486+0000 7f89ce3f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-25T08:41:08.483 INFO:tasks.ceph.mon.a.trial042.stderr:Please be aware that this feature is experimental, untested, 2026-02-25T08:41:08.483 INFO:tasks.ceph.mon.a.trial042.stderr:unsupported, and may result in data corruption, data loss, 2026-02-25T08:41:08.483 INFO:tasks.ceph.mon.a.trial042.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-25T08:41:08.483 INFO:tasks.ceph.mon.a.trial042.stderr:feature with important data. 2026-02-25T08:41:08.483 INFO:tasks.ceph.mon.a.trial042.stderr: 2026-02-25T08:41:09.049 INFO:tasks.ceph.mgr.x.trial066.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T08:41:09.050 INFO:tasks.ceph.mgr.x.trial066.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T08:41:09.050 INFO:tasks.ceph.mgr.x.trial066.stderr: from numpy import show_config as show_numpy_config 2026-02-25T08:41:09.373 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-25T08:41:09.373 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:41:09.374 DEBUG:teuthology.orchestra.run.trial042:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-25T08:41:09.399 DEBUG:teuthology.orchestra.run.trial042:> set -ex 2026-02-25T08:41:09.399 DEBUG:teuthology.orchestra.run.trial042:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-25T08:41:09.463 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:41:09.463 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-25T08:41:09.488 DEBUG:teuthology.orchestra.run.trial066:> set -ex 2026-02-25T08:41:09.488 DEBUG:teuthology.orchestra.run.trial066:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-25T08:41:09.553 DEBUG:teuthology.orchestra.run.trial066:> sudo ceph --cluster ceph osd new 75b5d2b2-3921-435a-8abf-38273b733491 0 2026-02-25T08:41:09.666 INFO:teuthology.orchestra.run.trial066.stderr:2026-02-25T08:41:09.662+0000 7efc31fa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.669 INFO:teuthology.orchestra.run.trial066.stderr:2026-02-25T08:41:09.668+0000 7efc31fa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.707 INFO:teuthology.orchestra.run.trial066.stdout:0 2026-02-25T08:41:09.714 DEBUG:teuthology.orchestra.run.trial066:> sudo ceph --cluster ceph osd new 9382ff8d-4393-4f8d-98fe-7efdc59fbafa 1 2026-02-25T08:41:09.789 INFO:teuthology.orchestra.run.trial066.stderr:2026-02-25T08:41:09.785+0000 7f100d88b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.791 INFO:teuthology.orchestra.run.trial066.stderr:2026-02-25T08:41:09.790+0000 7f100d88b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.828 INFO:teuthology.orchestra.run.trial066.stdout:1 2026-02-25T08:41:09.837 DEBUG:teuthology.orchestra.run.trial066:> sudo ceph --cluster ceph osd new 39831f49-8286-4170-a257-7a10577758c7 2 2026-02-25T08:41:09.910 INFO:teuthology.orchestra.run.trial066.stderr:2026-02-25T08:41:09.906+0000 7f378d1a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.913 INFO:teuthology.orchestra.run.trial066.stderr:2026-02-25T08:41:09.912+0000 7f378d1a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.950 INFO:teuthology.orchestra.run.trial066.stdout:2 2026-02-25T08:41:09.958 DEBUG:teuthology.orchestra.run.trial066:> sudo ceph --cluster ceph osd new eb0e4103-889c-4d45-b66b-668528a8f624 3 2026-02-25T08:41:10.031 INFO:teuthology.orchestra.run.trial066.stderr:2026-02-25T08:41:10.028+0000 7ff99b6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:10.033 INFO:teuthology.orchestra.run.trial066.stderr:2026-02-25T08:41:10.033+0000 7ff99b6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:10.071 INFO:teuthology.orchestra.run.trial066.stdout:3 2026-02-25T08:41:10.079 DEBUG:teuthology.orchestra.run.trial042:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-25T08:41:10.109 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-25T08:41:10.109 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-25T08:41:10.151 INFO:tasks.ceph.osd.0:Started 2026-02-25T08:41:10.151 DEBUG:teuthology.orchestra.run.trial042:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-25T08:41:10.179 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-25T08:41:10.179 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-25T08:41:10.221 INFO:tasks.ceph.osd.1:Started 2026-02-25T08:41:10.221 DEBUG:teuthology.orchestra.run.trial066:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-25T08:41:10.248 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-25T08:41:10.249 DEBUG:teuthology.orchestra.run.trial066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-25T08:41:10.291 INFO:tasks.ceph.osd.2:Started 2026-02-25T08:41:10.291 DEBUG:teuthology.orchestra.run.trial066:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2026-02-25T08:41:10.319 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-25T08:41:10.319 DEBUG:teuthology.orchestra.run.trial066:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2026-02-25T08:41:10.361 INFO:tasks.ceph.osd.3:Started 2026-02-25T08:41:10.362 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:10.438 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:10.439+0000 7f6e6a01a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:10.441 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:10.444+0000 7f6e6a01a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:10.473 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:10.474 INFO:teuthology.orchestra.run.trial042.stdout:{"epoch":7,"fsid":"517215f2-d42c-4e04-9968-4190617e61fc","created":"2026-02-25T08:41:08.204101+0000","modified":"2026-02-25T08:41:10.074242+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T08:41:10.074242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"75b5d2b2-3921-435a-8abf-38273b733491","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9382ff8d-4393-4f8d-98fe-7efdc59fbafa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"39831f49-8286-4170-a257-7a10577758c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eb0e4103-889c-4d45-b66b-668528a8f624","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:10.482 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-25T08:41:10.482 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-25T08:41:10.885 DEBUG:teuthology.orchestra.run.trial042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:10.951 INFO:teuthology.misc.health.trial042.stderr:2026-02-25T08:41:10.951+0000 7fa9eab93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:10.953 INFO:teuthology.misc.health.trial042.stderr:2026-02-25T08:41:10.956+0000 7fa9eab93640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:10.987 INFO:teuthology.misc.health.trial042.stdout: 2026-02-25T08:41:10.987 INFO:teuthology.misc.health.trial042.stdout:{"epoch":7,"fsid":"517215f2-d42c-4e04-9968-4190617e61fc","created":"2026-02-25T08:41:08.204101+0000","modified":"2026-02-25T08:41:10.074242+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T08:41:10.074242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"75b5d2b2-3921-435a-8abf-38273b733491","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9382ff8d-4393-4f8d-98fe-7efdc59fbafa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"39831f49-8286-4170-a257-7a10577758c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eb0e4103-889c-4d45-b66b-668528a8f624","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:10.993 DEBUG:teuthology.misc:0 of 4 OSDs are up 2026-02-25T08:41:11.419 INFO:tasks.ceph.mgr.x.trial066.stderr:2026-02-25T08:41:11.418+0000 7f4bbbb33640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-25T08:41:17.396 DEBUG:teuthology.orchestra.run.trial042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:17.466 INFO:teuthology.misc.health.trial042.stderr:2026-02-25T08:41:17.466+0000 7fe97a10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:17.469 INFO:teuthology.misc.health.trial042.stderr:2026-02-25T08:41:17.471+0000 7fe97a10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:17.585 INFO:teuthology.misc.health.trial042.stdout: 2026-02-25T08:41:17.585 INFO:teuthology.misc.health.trial042.stdout:{"epoch":10,"fsid":"517215f2-d42c-4e04-9968-4190617e61fc","created":"2026-02-25T08:41:08.204101+0000","modified":"2026-02-25T08:41:16.688769+0000","last_up_change":"2026-02-25T08:41:16.688769+0000","last_in_change":"2026-02-25T08:41:10.074242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"75b5d2b2-3921-435a-8abf-38273b733491","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9382ff8d-4393-4f8d-98fe-7efdc59fbafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6800","nonce":1687712840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6801","nonce":1687712840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6805","nonce":1687712840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6802","nonce":1687712840}]},"public_addr":"10.20.193.42:6800/1687712840","cluster_addr":"10.20.193.42:6801/1687712840","heartbeat_back_addr":"10.20.193.42:6805/1687712840","heartbeat_front_addr":"10.20.193.42:6802/1687712840","state":["exists","up"]},{"osd":2,"uuid":"39831f49-8286-4170-a257-7a10577758c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eb0e4103-889c-4d45-b66b-668528a8f624","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:17.593 DEBUG:teuthology.misc:1 of 4 OSDs are up 2026-02-25T08:41:23.997 DEBUG:teuthology.orchestra.run.trial042:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:24.067 INFO:teuthology.misc.health.trial042.stderr:2026-02-25T08:41:24.067+0000 7fc810e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:24.069 INFO:teuthology.misc.health.trial042.stderr:2026-02-25T08:41:24.072+0000 7fc810e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:24.183 INFO:teuthology.misc.health.trial042.stdout: 2026-02-25T08:41:24.184 INFO:teuthology.misc.health.trial042.stdout:{"epoch":14,"fsid":"517215f2-d42c-4e04-9968-4190617e61fc","created":"2026-02-25T08:41:08.204101+0000","modified":"2026-02-25T08:41:20.705832+0000","last_up_change":"2026-02-25T08:41:17.692407+0000","last_in_change":"2026-02-25T08:41:10.074242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T08:41:18.438046+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75b5d2b2-3921-435a-8abf-38273b733491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6804","nonce":3265065903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6803","nonce":3265065903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6807","nonce":3265065903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6806","nonce":3265065903}]},"public_addr":"10.20.193.42:6804/3265065903","cluster_addr":"10.20.193.42:6803/3265065903","heartbeat_back_addr":"10.20.193.42:6807/3265065903","heartbeat_front_addr":"10.20.193.42:6806/3265065903","state":["exists","up"]},{"osd":1,"uuid":"9382ff8d-4393-4f8d-98fe-7efdc59fbafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6800","nonce":1687712840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6801","nonce":1687712840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6805","nonce":1687712840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6802","nonce":1687712840}]},"public_addr":"10.20.193.42:6800/1687712840","cluster_addr":"10.20.193.42:6801/1687712840","heartbeat_back_addr":"10.20.193.42:6805/1687712840","heartbeat_front_addr":"10.20.193.42:6802/1687712840","state":["exists","up"]},{"osd":2,"uuid":"39831f49-8286-4170-a257-7a10577758c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6803","nonce":2704347279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6802","nonce":2704347279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6807","nonce":2704347279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6808","nonce":2704347279}]},"public_addr":"10.20.193.66:6803/2704347279","cluster_addr":"10.20.193.66:6802/2704347279","heartbeat_back_addr":"10.20.193.66:6807/2704347279","heartbeat_front_addr":"10.20.193.66:6808/2704347279","state":["exists","up"]},{"osd":3,"uuid":"eb0e4103-889c-4d45-b66b-668528a8f624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6806","nonce":3628997703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6805","nonce":3628997703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6809","nonce":3628997703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6810","nonce":3628997703}]},"public_addr":"10.20.193.66:6806/3628997703","cluster_addr":"10.20.193.66:6805/3628997703","heartbeat_back_addr":"10.20.193.66:6809/3628997703","heartbeat_front_addr":"10.20.193.66:6810/3628997703","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:24.191 DEBUG:teuthology.misc:4 of 4 OSDs are up 2026-02-25T08:41:24.192 INFO:tasks.ceph:Creating RBD pool 2026-02-25T08:41:24.192 DEBUG:teuthology.orchestra.run.trial042:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-25T08:41:24.268 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:24.266+0000 7f81c425a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:24.270 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:24.272+0000 7f81c425a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:25.239 INFO:teuthology.orchestra.run.trial042.stderr:pool 'rbd' created 2026-02-25T08:41:25.252 DEBUG:teuthology.orchestra.run.trial042:> rbd --cluster ceph pool init rbd 2026-02-25T08:41:25.291 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:25.293+0000 7fee1e06e240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:25.291 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:25.293+0000 7fee1e06e240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:25.294 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:25.296+0000 7fee1e06e240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.275 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-25T08:41:28.276 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-25T08:41:28.277 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-25T08:41:28.392 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:28.392+0000 7fb4c130b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.395 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:28.397+0000 7fb4c130b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.506 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:28.515 INFO:teuthology.orchestra.run.trial042.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-25T08:41:28.516 INFO:tasks.ceph_manager:config epoch is 1 2026-02-25T08:41:28.516 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-25T08:41:28.516 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-25T08:41:28.516 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-25T08:41:28.592 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:28.592+0000 7fd6f205a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.594 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:28.596+0000 7fd6f205a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.736 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:28.745 INFO:teuthology.orchestra.run.trial042.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6800","nonce":1003730052},{"type":"v1","addr":"10.20.193.66:6801","nonce":1003730052}]},"active_addr":"10.20.193.66:6801/1003730052","active_change":"2026-02-25T08:41:10.415258+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.66:0","nonce":2344442150}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.66:0","nonce":2032072536}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.66:0","nonce":1094153189}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.66:0","nonce":3848577160}]}]} 2026-02-25T08:41:28.749 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-25T08:41:28.749 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-25T08:41:28.749 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:28.824 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:28.824+0000 7f7f3c6c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.826 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:28.828+0000 7f7f3c6c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.945 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:28.945 INFO:teuthology.orchestra.run.trial042.stdout:{"epoch":18,"fsid":"517215f2-d42c-4e04-9968-4190617e61fc","created":"2026-02-25T08:41:08.204101+0000","modified":"2026-02-25T08:41:28.249019+0000","last_up_change":"2026-02-25T08:41:17.692407+0000","last_in_change":"2026-02-25T08:41:10.074242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T08:41:18.438046+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-25T08:41:24.388034+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75b5d2b2-3921-435a-8abf-38273b733491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6804","nonce":3265065903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6803","nonce":3265065903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6807","nonce":3265065903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6806","nonce":3265065903}]},"public_addr":"10.20.193.42:6804/3265065903","cluster_addr":"10.20.193.42:6803/3265065903","heartbeat_back_addr":"10.20.193.42:6807/3265065903","heartbeat_front_addr":"10.20.193.42:6806/3265065903","state":["exists","up"]},{"osd":1,"uuid":"9382ff8d-4393-4f8d-98fe-7efdc59fbafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6800","nonce":1687712840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6801","nonce":1687712840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6805","nonce":1687712840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6802","nonce":1687712840}]},"public_addr":"10.20.193.42:6800/1687712840","cluster_addr":"10.20.193.42:6801/1687712840","heartbeat_back_addr":"10.20.193.42:6805/1687712840","heartbeat_front_addr":"10.20.193.42:6802/1687712840","state":["exists","up"]},{"osd":2,"uuid":"39831f49-8286-4170-a257-7a10577758c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6803","nonce":2704347279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6802","nonce":2704347279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6807","nonce":2704347279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6808","nonce":2704347279}]},"public_addr":"10.20.193.66:6803/2704347279","cluster_addr":"10.20.193.66:6802/2704347279","heartbeat_back_addr":"10.20.193.66:6807/2704347279","heartbeat_front_addr":"10.20.193.66:6808/2704347279","state":["exists","up"]},{"osd":3,"uuid":"eb0e4103-889c-4d45-b66b-668528a8f624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6806","nonce":3628997703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6805","nonce":3628997703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6809","nonce":3628997703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6810","nonce":3628997703}]},"public_addr":"10.20.193.66:6806/3628997703","cluster_addr":"10.20.193.66:6805/3628997703","heartbeat_back_addr":"10.20.193.66:6809/3628997703","heartbeat_front_addr":"10.20.193.66:6810/3628997703","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:28.955 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-25T08:41:28.955 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:29.031 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.030+0000 7f732478c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.033 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.035+0000 7f732478c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.145 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:29.145 INFO:teuthology.orchestra.run.trial042.stdout:{"epoch":18,"fsid":"517215f2-d42c-4e04-9968-4190617e61fc","created":"2026-02-25T08:41:08.204101+0000","modified":"2026-02-25T08:41:28.249019+0000","last_up_change":"2026-02-25T08:41:17.692407+0000","last_in_change":"2026-02-25T08:41:10.074242+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T08:41:18.438046+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.75,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-25T08:41:24.388034+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"75b5d2b2-3921-435a-8abf-38273b733491","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6804","nonce":3265065903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6803","nonce":3265065903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6807","nonce":3265065903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6806","nonce":3265065903}]},"public_addr":"10.20.193.42:6804/3265065903","cluster_addr":"10.20.193.42:6803/3265065903","heartbeat_back_addr":"10.20.193.42:6807/3265065903","heartbeat_front_addr":"10.20.193.42:6806/3265065903","state":["exists","up"]},{"osd":1,"uuid":"9382ff8d-4393-4f8d-98fe-7efdc59fbafa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6800","nonce":1687712840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6801","nonce":1687712840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6805","nonce":1687712840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.42:6802","nonce":1687712840}]},"public_addr":"10.20.193.42:6800/1687712840","cluster_addr":"10.20.193.42:6801/1687712840","heartbeat_back_addr":"10.20.193.42:6805/1687712840","heartbeat_front_addr":"10.20.193.42:6802/1687712840","state":["exists","up"]},{"osd":2,"uuid":"39831f49-8286-4170-a257-7a10577758c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6803","nonce":2704347279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6802","nonce":2704347279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6807","nonce":2704347279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6808","nonce":2704347279}]},"public_addr":"10.20.193.66:6803/2704347279","cluster_addr":"10.20.193.66:6802/2704347279","heartbeat_back_addr":"10.20.193.66:6807/2704347279","heartbeat_front_addr":"10.20.193.66:6808/2704347279","state":["exists","up"]},{"osd":3,"uuid":"eb0e4103-889c-4d45-b66b-668528a8f624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6806","nonce":3628997703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6805","nonce":3628997703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6809","nonce":3628997703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.66:6810","nonce":3628997703}]},"public_addr":"10.20.193.66:6806/3628997703","cluster_addr":"10.20.193.66:6805/3628997703","heartbeat_back_addr":"10.20.193.66:6809/3628997703","heartbeat_front_addr":"10.20.193.66:6810/3628997703","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:29.156 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-25T08:41:29.156 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-25T08:41:29.157 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-25T08:41:29.157 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-25T08:41:29.240 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.238+0000 7fe8c93a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.240 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.238+0000 7f5448eb7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.240 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.239+0000 7fe5a6d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.241 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.240+0000 7fe1eb95c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.242 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.244+0000 7f5448eb7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.243 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.244+0000 7fe8c93a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.243 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.245+0000 7fe5a6d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.244 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.246+0000 7fe1eb95c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.260 INFO:teuthology.orchestra.run.trial042.stdout:42949672972 2026-02-25T08:41:29.260 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T08:41:29.261 INFO:teuthology.orchestra.run.trial042.stdout:47244640267 2026-02-25T08:41:29.262 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T08:41:29.263 INFO:teuthology.orchestra.run.trial042.stdout:47244640267 2026-02-25T08:41:29.263 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-25T08:41:29.264 INFO:teuthology.orchestra.run.trial042.stdout:47244640267 2026-02-25T08:41:29.265 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-25T08:41:29.382 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.382+0000 7fe2703fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.383 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.382+0000 7f3f47d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.383 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.382+0000 7fc74e7a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.384 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.387+0000 7fe2703fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.385 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.388+0000 7fc74e7a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.386 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.388+0000 7f3f47d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.424 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.422+0000 7f47ab25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.426 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:29.428+0000 7f47ab25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:29.501 INFO:teuthology.orchestra.run.trial042.stdout:47244640265 2026-02-25T08:41:29.502 INFO:teuthology.orchestra.run.trial042.stdout:42949672969 2026-02-25T08:41:29.509 INFO:teuthology.orchestra.run.trial042.stdout:47244640265 2026-02-25T08:41:29.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640265 for osd.0 2026-02-25T08:41:29.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672972 got 42949672969 for osd.1 2026-02-25T08:41:29.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640265 for osd.3 2026-02-25T08:41:29.544 INFO:teuthology.orchestra.run.trial042.stdout:47244640265 2026-02-25T08:41:29.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640265 for osd.2 2026-02-25T08:41:30.513 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T08:41:30.514 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T08:41:30.521 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-25T08:41:30.555 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-25T08:41:30.592 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:30.591+0000 7f69aa30b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:30.592 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:30.591+0000 7f745d85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:30.594 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:30.597+0000 7f69aa30b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:30.595 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:30.597+0000 7f745d85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:30.637 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:30.636+0000 7f5a843a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:30.637 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:30.636+0000 7f654d85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:30.639 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:30.642+0000 7f5a843a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:30.640 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:30.642+0000 7f654d85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:30.708 INFO:teuthology.orchestra.run.trial042.stdout:47244640265 2026-02-25T08:41:30.713 INFO:teuthology.orchestra.run.trial042.stdout:42949672969 2026-02-25T08:41:30.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640265 for osd.0 2026-02-25T08:41:30.726 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672972 got 42949672969 for osd.1 2026-02-25T08:41:30.754 INFO:teuthology.orchestra.run.trial042.stdout:47244640265 2026-02-25T08:41:30.756 INFO:teuthology.orchestra.run.trial042.stdout:47244640265 2026-02-25T08:41:30.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640265 for osd.2 2026-02-25T08:41:30.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640265 for osd.3 2026-02-25T08:41:31.721 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T08:41:31.726 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T08:41:31.765 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-25T08:41:31.767 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-25T08:41:31.800 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:31.799+0000 7f05b790b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:31.803 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:31.805+0000 7f05b790b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:31.844 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:31.844+0000 7f8a1b8c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:31.845 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:31.845+0000 7fc2bc4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:31.847 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:31.849+0000 7f8a1b8c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:31.847 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:31.850+0000 7fc2bc4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:31.878 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:31.877+0000 7f8ae5246640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:31.880 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:31.882+0000 7f8ae5246640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:31.917 INFO:teuthology.orchestra.run.trial042.stdout:47244640267 2026-02-25T08:41:31.930 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640267 for osd.0 2026-02-25T08:41:31.930 DEBUG:teuthology.parallel:result is None 2026-02-25T08:41:31.962 INFO:teuthology.orchestra.run.trial042.stdout:47244640267 2026-02-25T08:41:31.970 INFO:teuthology.orchestra.run.trial042.stdout:42949672972 2026-02-25T08:41:31.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640267 for osd.2 2026-02-25T08:41:31.972 DEBUG:teuthology.parallel:result is None 2026-02-25T08:41:31.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672972 got 42949672972 for osd.1 2026-02-25T08:41:31.981 DEBUG:teuthology.parallel:result is None 2026-02-25T08:41:31.993 INFO:teuthology.orchestra.run.trial042.stdout:47244640267 2026-02-25T08:41:32.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640267 got 47244640267 for osd.3 2026-02-25T08:41:32.004 DEBUG:teuthology.parallel:result is None 2026-02-25T08:41:32.004 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T08:41:32.004 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:41:32.121 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.121+0000 7fe04d05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.123 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.125+0000 7fe04d05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.235 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:32.235 INFO:teuthology.orchestra.run.trial042.stderr:dumped all 2026-02-25T08:41:32.244 INFO:teuthology.orchestra.run.trial042.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-25T08:41:30.416608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83890216,"kb_used_data":83890216,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125824920,"statfs":{"total":214748299264,"available":128844718080,"internally_reserved":0,"allocated":85903581184,"data_stored":85903581184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.178070"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257487+0000","last_change":"2026-02-25T08:41:28.261259+0000","last_active":"2026-02-25T08:41:28.257487+0000","last_peered":"2026-02-25T08:41:28.257487+0000","last_clean":"2026-02-25T08:41:28.257487+0000","last_became_active":"2026-02-25T08:41:26.297414+0000","last_became_peered":"2026-02-25T08:41:26.297414+0000","last_unstale":"2026-02-25T08:41:28.257487+0000","last_undegraded":"2026-02-25T08:41:28.257487+0000","last_fullsized":"2026-02-25T08:41:28.257487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.258724+0000","last_change":"2026-02-25T08:41:28.261605+0000","last_active":"2026-02-25T08:41:28.258724+0000","last_peered":"2026-02-25T08:41:28.258724+0000","last_clean":"2026-02-25T08:41:28.258724+0000","last_became_active":"2026-02-25T08:41:26.282840+0000","last_became_peered":"2026-02-25T08:41:26.282840+0000","last_unstale":"2026-02-25T08:41:28.258724+0000","last_undegraded":"2026-02-25T08:41:28.258724+0000","last_fullsized":"2026-02-25T08:41:28.258724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257779+0000","last_change":"2026-02-25T08:41:28.261618+0000","last_active":"2026-02-25T08:41:28.257779+0000","last_peered":"2026-02-25T08:41:28.257779+0000","last_clean":"2026-02-25T08:41:28.257779+0000","last_became_active":"2026-02-25T08:41:26.284252+0000","last_became_peered":"2026-02-25T08:41:26.284252+0000","last_unstale":"2026-02-25T08:41:28.257779+0000","last_undegraded":"2026-02-25T08:41:28.257779+0000","last_fullsized":"2026-02-25T08:41:28.257779+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.263550+0000","last_change":"2026-02-25T08:41:28.265491+0000","last_active":"2026-02-25T08:41:28.263550+0000","last_peered":"2026-02-25T08:41:28.263550+0000","last_clean":"2026-02-25T08:41:28.263550+0000","last_became_active":"2026-02-25T08:41:26.285926+0000","last_became_peered":"2026-02-25T08:41:26.285926+0000","last_unstale":"2026-02-25T08:41:28.263550+0000","last_undegraded":"2026-02-25T08:41:28.263550+0000","last_fullsized":"2026-02-25T08:41:28.263550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.261547+0000","last_change":"2026-02-25T08:41:28.261547+0000","last_active":"2026-02-25T08:41:28.261547+0000","last_peered":"2026-02-25T08:41:28.261547+0000","last_clean":"2026-02-25T08:41:28.261547+0000","last_became_active":"2026-02-25T08:41:26.280456+0000","last_became_peered":"2026-02-25T08:41:26.280456+0000","last_unstale":"2026-02-25T08:41:28.261547+0000","last_undegraded":"2026-02-25T08:41:28.261547+0000","last_fullsized":"2026-02-25T08:41:28.261547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.260909+0000","last_change":"2026-02-25T08:41:28.263380+0000","last_active":"2026-02-25T08:41:28.260909+0000","last_peered":"2026-02-25T08:41:28.260909+0000","last_clean":"2026-02-25T08:41:28.260909+0000","last_became_active":"2026-02-25T08:41:26.287664+0000","last_became_peered":"2026-02-25T08:41:26.287664+0000","last_unstale":"2026-02-25T08:41:28.260909+0000","last_undegraded":"2026-02-25T08:41:28.260909+0000","last_fullsized":"2026-02-25T08:41:28.260909+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257784+0000","last_change":"2026-02-25T08:41:28.261595+0000","last_active":"2026-02-25T08:41:28.257784+0000","last_peered":"2026-02-25T08:41:28.257784+0000","last_clean":"2026-02-25T08:41:28.257784+0000","last_became_active":"2026-02-25T08:41:26.301018+0000","last_became_peered":"2026-02-25T08:41:26.301018+0000","last_unstale":"2026-02-25T08:41:28.257784+0000","last_undegraded":"2026-02-25T08:41:28.257784+0000","last_fullsized":"2026-02-25T08:41:28.257784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257511+0000","last_change":"2026-02-25T08:41:28.261332+0000","last_active":"2026-02-25T08:41:28.257511+0000","last_peered":"2026-02-25T08:41:28.257511+0000","last_clean":"2026-02-25T08:41:28.257511+0000","last_became_active":"2026-02-25T08:41:26.283745+0000","last_became_peered":"2026-02-25T08:41:26.283745+0000","last_unstale":"2026-02-25T08:41:28.257511+0000","last_undegraded":"2026-02-25T08:41:28.257511+0000","last_fullsized":"2026-02-25T08:41:28.257511+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":91,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.256488+0000","last_change":"2026-02-25T08:41:19.740059+0000","last_active":"2026-02-25T08:41:28.256488+0000","last_peered":"2026-02-25T08:41:28.256488+0000","last_clean":"2026-02-25T08:41:28.256488+0000","last_became_active":"2026-02-25T08:41:19.730156+0000","last_became_peered":"2026-02-25T08:41:19.730156+0000","last_unstale":"2026-02-25T08:41:28.256488+0000","last_undegraded":"2026-02-25T08:41:28.256488+0000","last_fullsized":"2026-02-25T08:41:28.256488+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_clean_scrub_stamp":"2026-02-25T08:41:18.697053+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96633745408,"internally_reserved":0,"allocated":64427479040,"data_stored":64427479040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971972,"kb_used_data":20971972,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456812,"statfs":{"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972764,"kb_used_data":20972764,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456020,"statfs":{"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972724,"kb_used_data":20972724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456060,"statfs":{"total":53687074816,"available":32211005440,"internally_reserved":0,"allocated":21476069376,"data_stored":21476069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972756,"kb_used_data":20972756,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456028,"statfs":{"total":53687074816,"available":32210972672,"internally_reserved":0,"allocated":21476102144,"data_stored":21476102144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211005440,"internally_reserved":0,"allocated":21476069376,"data_stored":21476069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:41:32.246 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:41:32.321 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.321+0000 7f515943d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.323 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.325+0000 7f515943d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.435 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:32.436 INFO:teuthology.orchestra.run.trial042.stderr:dumped all 2026-02-25T08:41:32.444 INFO:teuthology.orchestra.run.trial042.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-25T08:41:32.416967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83890216,"kb_used_data":83890216,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125824920,"statfs":{"total":214748299264,"available":128844718080,"internally_reserved":0,"allocated":85903581184,"data_stored":85903581184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.177454"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257487+0000","last_change":"2026-02-25T08:41:28.261259+0000","last_active":"2026-02-25T08:41:28.257487+0000","last_peered":"2026-02-25T08:41:28.257487+0000","last_clean":"2026-02-25T08:41:28.257487+0000","last_became_active":"2026-02-25T08:41:26.297414+0000","last_became_peered":"2026-02-25T08:41:26.297414+0000","last_unstale":"2026-02-25T08:41:28.257487+0000","last_undegraded":"2026-02-25T08:41:28.257487+0000","last_fullsized":"2026-02-25T08:41:28.257487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.258724+0000","last_change":"2026-02-25T08:41:28.261605+0000","last_active":"2026-02-25T08:41:28.258724+0000","last_peered":"2026-02-25T08:41:28.258724+0000","last_clean":"2026-02-25T08:41:28.258724+0000","last_became_active":"2026-02-25T08:41:26.282840+0000","last_became_peered":"2026-02-25T08:41:26.282840+0000","last_unstale":"2026-02-25T08:41:28.258724+0000","last_undegraded":"2026-02-25T08:41:28.258724+0000","last_fullsized":"2026-02-25T08:41:28.258724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257779+0000","last_change":"2026-02-25T08:41:28.261618+0000","last_active":"2026-02-25T08:41:28.257779+0000","last_peered":"2026-02-25T08:41:28.257779+0000","last_clean":"2026-02-25T08:41:28.257779+0000","last_became_active":"2026-02-25T08:41:26.284252+0000","last_became_peered":"2026-02-25T08:41:26.284252+0000","last_unstale":"2026-02-25T08:41:28.257779+0000","last_undegraded":"2026-02-25T08:41:28.257779+0000","last_fullsized":"2026-02-25T08:41:28.257779+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.263550+0000","last_change":"2026-02-25T08:41:28.265491+0000","last_active":"2026-02-25T08:41:28.263550+0000","last_peered":"2026-02-25T08:41:28.263550+0000","last_clean":"2026-02-25T08:41:28.263550+0000","last_became_active":"2026-02-25T08:41:26.285926+0000","last_became_peered":"2026-02-25T08:41:26.285926+0000","last_unstale":"2026-02-25T08:41:28.263550+0000","last_undegraded":"2026-02-25T08:41:28.263550+0000","last_fullsized":"2026-02-25T08:41:28.263550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.261547+0000","last_change":"2026-02-25T08:41:28.261547+0000","last_active":"2026-02-25T08:41:28.261547+0000","last_peered":"2026-02-25T08:41:28.261547+0000","last_clean":"2026-02-25T08:41:28.261547+0000","last_became_active":"2026-02-25T08:41:26.280456+0000","last_became_peered":"2026-02-25T08:41:26.280456+0000","last_unstale":"2026-02-25T08:41:28.261547+0000","last_undegraded":"2026-02-25T08:41:28.261547+0000","last_fullsized":"2026-02-25T08:41:28.261547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.260909+0000","last_change":"2026-02-25T08:41:28.263380+0000","last_active":"2026-02-25T08:41:28.260909+0000","last_peered":"2026-02-25T08:41:28.260909+0000","last_clean":"2026-02-25T08:41:28.260909+0000","last_became_active":"2026-02-25T08:41:26.287664+0000","last_became_peered":"2026-02-25T08:41:26.287664+0000","last_unstale":"2026-02-25T08:41:28.260909+0000","last_undegraded":"2026-02-25T08:41:28.260909+0000","last_fullsized":"2026-02-25T08:41:28.260909+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257784+0000","last_change":"2026-02-25T08:41:28.261595+0000","last_active":"2026-02-25T08:41:28.257784+0000","last_peered":"2026-02-25T08:41:28.257784+0000","last_clean":"2026-02-25T08:41:28.257784+0000","last_became_active":"2026-02-25T08:41:26.301018+0000","last_became_peered":"2026-02-25T08:41:26.301018+0000","last_unstale":"2026-02-25T08:41:28.257784+0000","last_undegraded":"2026-02-25T08:41:28.257784+0000","last_fullsized":"2026-02-25T08:41:28.257784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257511+0000","last_change":"2026-02-25T08:41:28.261332+0000","last_active":"2026-02-25T08:41:28.257511+0000","last_peered":"2026-02-25T08:41:28.257511+0000","last_clean":"2026-02-25T08:41:28.257511+0000","last_became_active":"2026-02-25T08:41:26.283745+0000","last_became_peered":"2026-02-25T08:41:26.283745+0000","last_unstale":"2026-02-25T08:41:28.257511+0000","last_undegraded":"2026-02-25T08:41:28.257511+0000","last_fullsized":"2026-02-25T08:41:28.257511+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":91,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.256488+0000","last_change":"2026-02-25T08:41:19.740059+0000","last_active":"2026-02-25T08:41:28.256488+0000","last_peered":"2026-02-25T08:41:28.256488+0000","last_clean":"2026-02-25T08:41:28.256488+0000","last_became_active":"2026-02-25T08:41:19.730156+0000","last_became_peered":"2026-02-25T08:41:19.730156+0000","last_unstale":"2026-02-25T08:41:28.256488+0000","last_undegraded":"2026-02-25T08:41:28.256488+0000","last_fullsized":"2026-02-25T08:41:28.256488+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_clean_scrub_stamp":"2026-02-25T08:41:18.697053+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96633745408,"internally_reserved":0,"allocated":64427479040,"data_stored":64427479040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971972,"kb_used_data":20971972,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456812,"statfs":{"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972764,"kb_used_data":20972764,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456020,"statfs":{"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972724,"kb_used_data":20972724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456060,"statfs":{"total":53687074816,"available":32211005440,"internally_reserved":0,"allocated":21476069376,"data_stored":21476069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972756,"kb_used_data":20972756,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456028,"statfs":{"total":53687074816,"available":32210972672,"internally_reserved":0,"allocated":21476102144,"data_stored":21476102144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211005440,"internally_reserved":0,"allocated":21476069376,"data_stored":21476069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:41:32.446 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T08:41:32.446 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-25T08:41:32.446 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-25T08:41:32.446 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-25T08:41:32.522 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.522+0000 7f0deb21a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.524 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.526+0000 7f0deb21a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.662 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:32.662 INFO:teuthology.orchestra.run.trial042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-25T08:41:32.671 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-25T08:41:32.671 INFO:teuthology.run_tasks:Running task thrashosds... 2026-02-25T08:41:32.683 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'chance_trim_stale_osdmaps': 0, 'dump_ops_enable': False, 'min_in': 3, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2026-02-25T08:41:32.684 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-logstoremanager'}, 'ceph': {'conf': {'client.0': {'admin socket': '/var/run/ceph/ceph-$name.asok'}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'osd pool default crimson': True, 'paxos service trim min': 10}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd objectstore': 'seastore', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2, 'osd target transaction size': 5, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'Degraded data redundancy', 'is down', 'osds down', 'pg .*? is .*?degraded.*?, acting', 'pg .*? is stuck', 'pg degraded', 'PG_BACKFILL_FULL', 'Low space hindering backfill .*? backfill_toofull', 'OSD_ROOT_DOWN', 'pgs degraded', 'pgs undersized', 'is active.*backfill_toofull.*'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, 'thrashosds': {'min_in': 4}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d'}} 2026-02-25T08:41:32.684 INFO:tasks.thrashosds:config is {'ceph_objectstore_tool': False, 'chance_inject_pause_short': 0, 'chance_reset_purged_snaps_last': 0, 'chance_thrash_cluster_full': 0, 'chance_thrash_pg_upmap': 0, 'chance_trim_stale_osdmaps': 0, 'dump_ops_enable': False, 'min_in': 4, 'noscrub_toggle_delay': 0, 'reweight_osd': 0, 'sighup_delay': 0, 'thrash_primary_affinity': False, 'timeout': 2400, 'optrack_toggle_delay': 2.0, 'random_eio': 0.0} 2026-02-25T08:41:32.684 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2026-02-25T08:41:32.761 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.759+0000 7f402228b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.763 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.765+0000 7f402228b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.886 INFO:tasks.thrashosds:Beginning thrashosds... 2026-02-25T08:41:32.886 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T08:41:32.886 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:41:32.961 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.961+0000 7f331efb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:32.963 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:32.966+0000 7f331efb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.073 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:33.073 INFO:teuthology.orchestra.run.trial042.stderr:dumped all 2026-02-25T08:41:33.082 INFO:teuthology.orchestra.run.trial042.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-25T08:41:32.416967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83890216,"kb_used_data":83890216,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125824920,"statfs":{"total":214748299264,"available":128844718080,"internally_reserved":0,"allocated":85903581184,"data_stored":85903581184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.177454"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257487+0000","last_change":"2026-02-25T08:41:28.261259+0000","last_active":"2026-02-25T08:41:28.257487+0000","last_peered":"2026-02-25T08:41:28.257487+0000","last_clean":"2026-02-25T08:41:28.257487+0000","last_became_active":"2026-02-25T08:41:26.297414+0000","last_became_peered":"2026-02-25T08:41:26.297414+0000","last_unstale":"2026-02-25T08:41:28.257487+0000","last_undegraded":"2026-02-25T08:41:28.257487+0000","last_fullsized":"2026-02-25T08:41:28.257487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.258724+0000","last_change":"2026-02-25T08:41:28.261605+0000","last_active":"2026-02-25T08:41:28.258724+0000","last_peered":"2026-02-25T08:41:28.258724+0000","last_clean":"2026-02-25T08:41:28.258724+0000","last_became_active":"2026-02-25T08:41:26.282840+0000","last_became_peered":"2026-02-25T08:41:26.282840+0000","last_unstale":"2026-02-25T08:41:28.258724+0000","last_undegraded":"2026-02-25T08:41:28.258724+0000","last_fullsized":"2026-02-25T08:41:28.258724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257779+0000","last_change":"2026-02-25T08:41:28.261618+0000","last_active":"2026-02-25T08:41:28.257779+0000","last_peered":"2026-02-25T08:41:28.257779+0000","last_clean":"2026-02-25T08:41:28.257779+0000","last_became_active":"2026-02-25T08:41:26.284252+0000","last_became_peered":"2026-02-25T08:41:26.284252+0000","last_unstale":"2026-02-25T08:41:28.257779+0000","last_undegraded":"2026-02-25T08:41:28.257779+0000","last_fullsized":"2026-02-25T08:41:28.257779+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.263550+0000","last_change":"2026-02-25T08:41:28.265491+0000","last_active":"2026-02-25T08:41:28.263550+0000","last_peered":"2026-02-25T08:41:28.263550+0000","last_clean":"2026-02-25T08:41:28.263550+0000","last_became_active":"2026-02-25T08:41:26.285926+0000","last_became_peered":"2026-02-25T08:41:26.285926+0000","last_unstale":"2026-02-25T08:41:28.263550+0000","last_undegraded":"2026-02-25T08:41:28.263550+0000","last_fullsized":"2026-02-25T08:41:28.263550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.261547+0000","last_change":"2026-02-25T08:41:28.261547+0000","last_active":"2026-02-25T08:41:28.261547+0000","last_peered":"2026-02-25T08:41:28.261547+0000","last_clean":"2026-02-25T08:41:28.261547+0000","last_became_active":"2026-02-25T08:41:26.280456+0000","last_became_peered":"2026-02-25T08:41:26.280456+0000","last_unstale":"2026-02-25T08:41:28.261547+0000","last_undegraded":"2026-02-25T08:41:28.261547+0000","last_fullsized":"2026-02-25T08:41:28.261547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.260909+0000","last_change":"2026-02-25T08:41:28.263380+0000","last_active":"2026-02-25T08:41:28.260909+0000","last_peered":"2026-02-25T08:41:28.260909+0000","last_clean":"2026-02-25T08:41:28.260909+0000","last_became_active":"2026-02-25T08:41:26.287664+0000","last_became_peered":"2026-02-25T08:41:26.287664+0000","last_unstale":"2026-02-25T08:41:28.260909+0000","last_undegraded":"2026-02-25T08:41:28.260909+0000","last_fullsized":"2026-02-25T08:41:28.260909+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257784+0000","last_change":"2026-02-25T08:41:28.261595+0000","last_active":"2026-02-25T08:41:28.257784+0000","last_peered":"2026-02-25T08:41:28.257784+0000","last_clean":"2026-02-25T08:41:28.257784+0000","last_became_active":"2026-02-25T08:41:26.301018+0000","last_became_peered":"2026-02-25T08:41:26.301018+0000","last_unstale":"2026-02-25T08:41:28.257784+0000","last_undegraded":"2026-02-25T08:41:28.257784+0000","last_fullsized":"2026-02-25T08:41:28.257784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257511+0000","last_change":"2026-02-25T08:41:28.261332+0000","last_active":"2026-02-25T08:41:28.257511+0000","last_peered":"2026-02-25T08:41:28.257511+0000","last_clean":"2026-02-25T08:41:28.257511+0000","last_became_active":"2026-02-25T08:41:26.283745+0000","last_became_peered":"2026-02-25T08:41:26.283745+0000","last_unstale":"2026-02-25T08:41:28.257511+0000","last_undegraded":"2026-02-25T08:41:28.257511+0000","last_fullsized":"2026-02-25T08:41:28.257511+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":91,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.256488+0000","last_change":"2026-02-25T08:41:19.740059+0000","last_active":"2026-02-25T08:41:28.256488+0000","last_peered":"2026-02-25T08:41:28.256488+0000","last_clean":"2026-02-25T08:41:28.256488+0000","last_became_active":"2026-02-25T08:41:19.730156+0000","last_became_peered":"2026-02-25T08:41:19.730156+0000","last_unstale":"2026-02-25T08:41:28.256488+0000","last_undegraded":"2026-02-25T08:41:28.256488+0000","last_fullsized":"2026-02-25T08:41:28.256488+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_clean_scrub_stamp":"2026-02-25T08:41:18.697053+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96633745408,"internally_reserved":0,"allocated":64427479040,"data_stored":64427479040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971972,"kb_used_data":20971972,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456812,"statfs":{"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972764,"kb_used_data":20972764,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456020,"statfs":{"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972724,"kb_used_data":20972724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456060,"statfs":{"total":53687074816,"available":32211005440,"internally_reserved":0,"allocated":21476069376,"data_stored":21476069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972756,"kb_used_data":20972756,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456028,"statfs":{"total":53687074816,"available":32210972672,"internally_reserved":0,"allocated":21476102144,"data_stored":21476102144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211005440,"internally_reserved":0,"allocated":21476069376,"data_stored":21476069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:41:33.083 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:41:33.157 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.157+0000 7fa3f9da4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.160 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.162+0000 7fa3f9da4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.270 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:33.271 INFO:teuthology.orchestra.run.trial042.stderr:dumped all 2026-02-25T08:41:33.280 INFO:teuthology.orchestra.run.trial042.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2026-02-25T08:41:32.416967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83890216,"kb_used_data":83890216,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125824920,"statfs":{"total":214748299264,"available":128844718080,"internally_reserved":0,"allocated":85903581184,"data_stored":85903581184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.177454"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257487+0000","last_change":"2026-02-25T08:41:28.261259+0000","last_active":"2026-02-25T08:41:28.257487+0000","last_peered":"2026-02-25T08:41:28.257487+0000","last_clean":"2026-02-25T08:41:28.257487+0000","last_became_active":"2026-02-25T08:41:26.297414+0000","last_became_peered":"2026-02-25T08:41:26.297414+0000","last_unstale":"2026-02-25T08:41:28.257487+0000","last_undegraded":"2026-02-25T08:41:28.257487+0000","last_fullsized":"2026-02-25T08:41:28.257487+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.258724+0000","last_change":"2026-02-25T08:41:28.261605+0000","last_active":"2026-02-25T08:41:28.258724+0000","last_peered":"2026-02-25T08:41:28.258724+0000","last_clean":"2026-02-25T08:41:28.258724+0000","last_became_active":"2026-02-25T08:41:26.282840+0000","last_became_peered":"2026-02-25T08:41:26.282840+0000","last_unstale":"2026-02-25T08:41:28.258724+0000","last_undegraded":"2026-02-25T08:41:28.258724+0000","last_fullsized":"2026-02-25T08:41:28.258724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257779+0000","last_change":"2026-02-25T08:41:28.261618+0000","last_active":"2026-02-25T08:41:28.257779+0000","last_peered":"2026-02-25T08:41:28.257779+0000","last_clean":"2026-02-25T08:41:28.257779+0000","last_became_active":"2026-02-25T08:41:26.284252+0000","last_became_peered":"2026-02-25T08:41:26.284252+0000","last_unstale":"2026-02-25T08:41:28.257779+0000","last_undegraded":"2026-02-25T08:41:28.257779+0000","last_fullsized":"2026-02-25T08:41:28.257779+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.263550+0000","last_change":"2026-02-25T08:41:28.265491+0000","last_active":"2026-02-25T08:41:28.263550+0000","last_peered":"2026-02-25T08:41:28.263550+0000","last_clean":"2026-02-25T08:41:28.263550+0000","last_became_active":"2026-02-25T08:41:26.285926+0000","last_became_peered":"2026-02-25T08:41:26.285926+0000","last_unstale":"2026-02-25T08:41:28.263550+0000","last_undegraded":"2026-02-25T08:41:28.263550+0000","last_fullsized":"2026-02-25T08:41:28.263550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.261547+0000","last_change":"2026-02-25T08:41:28.261547+0000","last_active":"2026-02-25T08:41:28.261547+0000","last_peered":"2026-02-25T08:41:28.261547+0000","last_clean":"2026-02-25T08:41:28.261547+0000","last_became_active":"2026-02-25T08:41:26.280456+0000","last_became_peered":"2026-02-25T08:41:26.280456+0000","last_unstale":"2026-02-25T08:41:28.261547+0000","last_undegraded":"2026-02-25T08:41:28.261547+0000","last_fullsized":"2026-02-25T08:41:28.261547+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.260909+0000","last_change":"2026-02-25T08:41:28.263380+0000","last_active":"2026-02-25T08:41:28.260909+0000","last_peered":"2026-02-25T08:41:28.260909+0000","last_clean":"2026-02-25T08:41:28.260909+0000","last_became_active":"2026-02-25T08:41:26.287664+0000","last_became_peered":"2026-02-25T08:41:26.287664+0000","last_unstale":"2026-02-25T08:41:28.260909+0000","last_undegraded":"2026-02-25T08:41:28.260909+0000","last_fullsized":"2026-02-25T08:41:28.260909+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257784+0000","last_change":"2026-02-25T08:41:28.261595+0000","last_active":"2026-02-25T08:41:28.257784+0000","last_peered":"2026-02-25T08:41:28.257784+0000","last_clean":"2026-02-25T08:41:28.257784+0000","last_became_active":"2026-02-25T08:41:26.301018+0000","last_became_peered":"2026-02-25T08:41:26.301018+0000","last_unstale":"2026-02-25T08:41:28.257784+0000","last_undegraded":"2026-02-25T08:41:28.257784+0000","last_fullsized":"2026-02-25T08:41:28.257784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.257511+0000","last_change":"2026-02-25T08:41:28.261332+0000","last_active":"2026-02-25T08:41:28.257511+0000","last_peered":"2026-02-25T08:41:28.257511+0000","last_clean":"2026-02-25T08:41:28.257511+0000","last_became_active":"2026-02-25T08:41:26.283745+0000","last_became_peered":"2026-02-25T08:41:26.283745+0000","last_unstale":"2026-02-25T08:41:28.257511+0000","last_undegraded":"2026-02-25T08:41:28.257511+0000","last_fullsized":"2026-02-25T08:41:28.257511+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":91,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.256488+0000","last_change":"2026-02-25T08:41:19.740059+0000","last_active":"2026-02-25T08:41:28.256488+0000","last_peered":"2026-02-25T08:41:28.256488+0000","last_clean":"2026-02-25T08:41:28.256488+0000","last_became_active":"2026-02-25T08:41:19.730156+0000","last_became_peered":"2026-02-25T08:41:19.730156+0000","last_unstale":"2026-02-25T08:41:28.256488+0000","last_undegraded":"2026-02-25T08:41:28.256488+0000","last_fullsized":"2026-02-25T08:41:28.256488+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_clean_scrub_stamp":"2026-02-25T08:41:18.697053+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96633745408,"internally_reserved":0,"allocated":64427479040,"data_stored":64427479040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":24,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971972,"kb_used_data":20971972,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456812,"statfs":{"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972764,"kb_used_data":20972764,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456020,"statfs":{"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972724,"kb_used_data":20972724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456060,"statfs":{"total":53687074816,"available":32211005440,"internally_reserved":0,"allocated":21476069376,"data_stored":21476069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640267,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972756,"kb_used_data":20972756,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456028,"statfs":{"total":53687074816,"available":32210972672,"internally_reserved":0,"allocated":21476102144,"data_stored":21476102144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211005440,"internally_reserved":0,"allocated":21476069376,"data_stored":21476069376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32210964480,"internally_reserved":0,"allocated":21476110336,"data_stored":21476110336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:41:33.281 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T08:41:33.281 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2026-02-25T08:41:33.356 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.356+0000 7f9e5090b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.359 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.361+0000 7f9e5090b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:epoch 18 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:fsid 517215f2-d42c-4e04-9968-4190617e61fc 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:created 2026-02-25T08:41:08.204101+0000 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:modified 2026-02-25T08:41:28.249019+0000 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:crush_version 5 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:full_ratio 0.95 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:backfillfull_ratio 0.9 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:nearfull_ratio 0.85 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:require_min_compat_client luminous 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:min_compat_client jewel 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:require_osd_release tentacle 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:stretch_mode_enabled false 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:allow_crimson=true 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 14 flags hashpspool,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 4.00 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 18 flags hashpspool,selfmanaged_snaps,crimson stripe_width 0 application rbd read_balance_score 2.50 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout: removed_snaps_queue [2~1] 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:max_osd 4 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) v2:10.20.193.42:6804/3265065903 v2:10.20.193.42:6803/3265065903 exists,up 75b5d2b2-3921-435a-8abf-38273b733491 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:osd.1 up in weight 1 up_from 10 up_thru 15 down_at 0 last_clean_interval [0,0) v2:10.20.193.42:6800/1687712840 v2:10.20.193.42:6801/1687712840 exists,up 9382ff8d-4393-4f8d-98fe-7efdc59fbafa 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) v2:10.20.193.66:6803/2704347279 v2:10.20.193.66:6802/2704347279 exists,up 39831f49-8286-4170-a257-7a10577758c7 2026-02-25T08:41:33.474 INFO:teuthology.orchestra.run.trial042.stdout:osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) v2:10.20.193.66:6806/3628997703 v2:10.20.193.66:6805/3628997703 exists,up eb0e4103-889c-4d45-b66b-668528a8f624 2026-02-25T08:41:33.484 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 11 up_thru 0 down_at 0 last_clean_interval [0,0) v2:10.20.193.42:6804/3265065903 v2:10.20.193.42:6803/3265065903 exists,up 75b5d2b2-3921-435a-8abf-38273b733491', 'osd.1 up in weight 1 up_from 10 up_thru 15 down_at 0 last_clean_interval [0,0) v2:10.20.193.42:6800/1687712840 v2:10.20.193.42:6801/1687712840 exists,up 9382ff8d-4393-4f8d-98fe-7efdc59fbafa', 'osd.2 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) v2:10.20.193.66:6803/2704347279 v2:10.20.193.66:6802/2704347279 exists,up 39831f49-8286-4170-a257-7a10577758c7', 'osd.3 up in weight 1 up_from 11 up_thru 15 down_at 0 last_clean_interval [0,0) v2:10.20.193.66:6806/3628997703 v2:10.20.193.66:6805/3628997703 exists,up eb0e4103-889c-4d45-b66b-668528a8f624'] 2026-02-25T08:41:33.484 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2026-02-25T08:41:33.558 INFO:teuthology.orchestra.run.trial042.stdout:{ 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "name": "mon.a", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "cluster": "ceph", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "admin_socket_mode": "", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "auth_client_required": "cephx, none", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "auth_cluster_required": "cephx", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "auth_debug": "true", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2026-02-25T08:41:33.559 INFO:teuthology.orchestra.run.trial042.stdout: "auth_mon_ticket_ttl": "660.000000", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "auth_service_required": "cephx", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "auth_service_ticket_ttl": "240.000000", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "auth_supported": "cephx", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_aio": "true", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_aio_max_queue_depth": "1024", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_aio_poll_ms": "250", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_aio_reap_max": "16", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_aio_submit_retry_initial_delay_us": "125", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_aio_submit_retry_max": "16", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_async_discard": "false", 2026-02-25T08:41:33.560 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_async_discard_max_pending": "1000000", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_async_discard_threads": "0", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_block_size": "4096", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_debug_aio": "false", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_debug_aio_log_age": "5.000000", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_debug_discard_sleep": "0", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_debug_inflight_ios": "false", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_discard_max_bytes": "10737418240", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_enable_discard": "false", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_flock_retry": "3", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_flock_retry_interval": "0.100000", 2026-02-25T08:41:33.561 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_inject_crash": "0", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_inject_crash_flush_delay": "2", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_ioring": "false", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_ioring_hipri": "false", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_ioring_sqthread_poll": "false", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_max_discard_length": "2147483648", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_nvme_unbind_from_kernel": "false", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_read_buffer_alignment": "4096", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_read_preallocated_huge_buffers": "", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_stalled_read_warn_threshold": "1", 2026-02-25T08:41:33.562 INFO:teuthology.orchestra.run.trial042.stdout: "bdev_type": "", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_alloc_size": "1048576", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_allocator": "hybrid", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_buffered_io": "true", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_check_for_zeros": "false", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_check_volume_selector_often": "false", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_check_volume_selector_on_umount": "false", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_compact_log_sync": "false", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_log_compact_min_size": "16777216", 2026-02-25T08:41:33.563 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_log_replay_check_allocations": "true", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_max_log_runway": "4194304", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_max_prefetch": "1048576", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_min_flush_size": "524288", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_min_log_runway": "1048576", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_replay_recovery": "false", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_replay_recovery_disable_compact": "false", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_shared_alloc_size": "65536", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_sync_write": "false", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluefs_wal_envelope_mode": "true", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2026-02-25T08:41:33.564 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_allocation_from_file": "true", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_allocator": "hybrid", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_allocator_lookup_policy": "auto", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_async_db_compaction": "true", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bdev_label_multi": "true", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bdev_label_require_all": "true", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2026-02-25T08:41:33.565 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bitmapallocator_span_size": "1024", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_blobid_prealloc": "10240", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_create": "true", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_db_create": "false", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_db_path": "", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_db_size": "0", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_path": "", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_preallocate_file": "false", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_size": "107374182400", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_wal_create": "false", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_wal_path": "", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_block_wal_size": "100663296", 2026-02-25T08:41:33.566 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bluefs": "true", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bluefs_env_mirror": "false", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_bluefs_max_free": "10737418240", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_autotune": "true", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_autotune_interval": "5.000000", 2026-02-25T08:41:33.567 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_kv_ratio": "0.450000", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_meta_ratio": "0.450000", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_size": "0", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_size_hdd": "1073741824", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_size_ssd": "3221225472", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_trim_interval": "0.050000", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cache_type": "2q", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_clone_cow": "true", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_algorithm": "snappy", 2026-02-25T08:41:33.568 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_max_blob_size": "0", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_min_blob_size": "0", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_mode": "none", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_compression_required_ratio": "0.875000", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_csum_type": "crc32c", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_enforce_settings": "default", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_extent_map_encode_check": "false", 2026-02-25T08:41:33.569 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_freelist": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_fsck_abort": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_inject_read_err": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_legacy_omap": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_no_reuse_blocks": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_omit_block_device_write": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_omit_kv_commit": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_onode_segmentation_random": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_prefragment_max": "1048576", 2026-02-25T08:41:33.570 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_random_read_err": "0.000000", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_small_allocations": "0", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_default_buffered_read": "true", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_default_buffered_write": "false", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_deferred_batch_ops": "0", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_discard_on_mkfs": "true", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_elastic_shared_blobs": "true", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2026-02-25T08:41:33.571 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_extent_map_shard_max_size": "1200", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_extent_map_shard_min_size": "150", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_extent_map_shard_target_size": "500", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fail_eio": "false", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fragmentation_check_period": "3600", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_freelist_blocks_per_key": "128", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_on_mkfs": "true", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2026-02-25T08:41:33.572 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_on_mount": "false", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_on_mount_deep": "false", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_on_umount": "false", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_on_umount_deep": "false", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_quick_fix_threads": "2", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_gc_enable_blob_threshold": "0", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_gc_enable_total_threshold": "0", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_ignore_data_csum": "false", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2026-02-25T08:41:33.573 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_kvbackend": "rocksdb", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_log_collection_list_age": "60.000000", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_log_op_age": "5.000000", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_max_alloc_size": "0", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_max_blob_size": "0", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_max_blob_size_hdd": "65536", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_max_blob_size_ssd": "65536", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_max_defer_interval": "3.000000", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_max_deferred_txc": "32", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_min_alloc_size": "0", 2026-02-25T08:41:33.574 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_min_alloc_size_hdd": "4096", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_min_alloc_size_ssd": "4096", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_nid_prealloc": "1024", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_onode_segment_size": "0", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_prefer_deferred_size": "0", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_qfsck_on_mount": "true", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_recompression_min_gain": "1.200000", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_retry_disk_reads": "3", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_rocksdb_cf": "true", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2026-02-25T08:41:33.575 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_rocksdb_options_annex": "", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_slow_ops_warn_threshold": "1", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_spdk_coremask": "0x1", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_spdk_io_sleep": "5", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_spdk_max_io_completion": "0", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_spdk_mem": "512", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_sync_submit_transaction": "false", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_throttle_bytes": "67108864", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_throttle_cost_per_io": "0", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2026-02-25T08:41:33.576 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_throttle_trace_rate": "0.000000", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_tracing": "false", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_volume_selection_reserved": "0", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_warn_on_free_fragmentation": "0.800000", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_warn_on_legacy_statfs": "true", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2026-02-25T08:41:33.577 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_write_v2": "false", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_write_v2_random": "false", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "bluestore_zero_block_detection": "false", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "breakpad": "false", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "ceph_assert_supresssions": "", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "cephadm_path": "/usr/sbin/cephadm", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_action_update_interval": "2", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_directory_scan_interval": "10", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2026-02-25T08:41:33.578 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_mount_timeout": "10", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_perf_stats_prio": "5", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephsqlite_blocklist_dead_locker": "true", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephsqlite_lock_renewal_interval": "2000", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephx_cluster_require_signatures": "false", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephx_cluster_require_version": "2", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephx_require_signatures": "false", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephx_require_version": "2", 2026-02-25T08:41:33.579 INFO:teuthology.orchestra.run.trial042.stdout: "cephx_service_require_signatures": "false", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "cephx_service_require_version": "2", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "cephx_sign_messages": "true", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "chdir": "", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_acl_type": "", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_asio_thread_count": "2", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_cache_mid": "0.750000", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_cache_size": "16384", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_caps_release_delay": "5", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_check_pool_perm": "true", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_collect_and_send_global_metrics": "false", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_debug_force_sync_read": "false", 2026-02-25T08:41:33.580 INFO:teuthology.orchestra.run.trial042.stdout: "client_debug_getattr_caps": "false", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_debug_inject_features": "", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_debug_inject_tick_delay": "0", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_die_on_failed_dentry_invalidate": "true", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_die_on_failed_remount": "false", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_dirsize_rbytes": "true", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_file_blockdiff_max_concurrent_object_scans": "16", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_force_lazyio": "false", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_fs": "", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_fscrypt_as": "true", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_fscrypt_dummy_encryption": "false", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_inject_fixed_oldest_tid": "false", 2026-02-25T08:41:33.581 INFO:teuthology.orchestra.run.trial042.stdout: "client_inject_release_failure": "false", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_inject_write_delay_secs": "0", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_max_inline_size": "4096", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_max_retries_on_remount_failure": "5", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_mds_namespace": "", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_metadata": "", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_mount_gid": "-1", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_mount_timeout": "300", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_mount_uid": "-1", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_mountpoint": "/", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_notify_timeout": "10", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_oc": "true", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_oc_max_dirty": "104857600", 2026-02-25T08:41:33.582 INFO:teuthology.orchestra.run.trial042.stdout: "client_oc_max_dirty_age": "5.000000", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_oc_max_objects": "1000", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_oc_size": "209715200", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_oc_target_dirty": "8388608", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_permissions": "true", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_quota": "true", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_quota_df": "true", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_readahead_max_bytes": "0", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_readahead_max_periods": "4", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_readahead_min": "131072", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_reconnect_stale": "false", 2026-02-25T08:41:33.583 INFO:teuthology.orchestra.run.trial042.stdout: "client_respect_subvolume_snapshot_visibility": "false", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "client_shutdown_timeout": "30", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "client_snapdir": ".snap", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "client_tick_interval": "1", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "client_trace": "", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "client_try_dentry_invalidate": "false", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "client_use_faked_inos": "false", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "client_use_random_mds": "false", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "clog_to_graylog": "false", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "clog_to_graylog_host": "127.0.0.1", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "clog_to_graylog_port": "12201", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "clog_to_monitors": "default=true", 2026-02-25T08:41:33.584 INFO:teuthology.orchestra.run.trial042.stdout: "clog_to_syslog": "false", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "clog_to_syslog_level": "info", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "cluster_addr": "-", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "cluster_network": "", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "cluster_network_interface": "", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "compressor_zlib_isal": "false", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "compressor_zlib_level": "5", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "compressor_zlib_winsize": "-15", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "compressor_zstd_level": "1", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "crash_dir": "/var/lib/ceph/crash", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_allow_pg_split": "true", 2026-02-25T08:41:33.585 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_bluestore_cpu_set": "", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_bluestore_num_threads": "6", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_cpu_num": "0", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_cpu_set": "", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_osd_obc_lru_size": "512", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_osd_scheduler_concurrency": "0", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_osd_stat_interval": "0", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_poll_mode": "false", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_reactor_idle_poll_time_us": "200", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_reactor_io_latency_goal_ms": "0.000000", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crimson_reactor_task_quota_ms": "0.500000", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crush_location": "", 2026-02-25T08:41:33.586 INFO:teuthology.orchestra.run.trial042.stdout: "crush_location_hook": "", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "crush_location_hook_timeout": "10", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "d4n_writecache_enabled": "false", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "daemonize": "false", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "daos_pool": "tank", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "dbstore_db_name_prefix": "dbstore", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "debug_alienstore": "0/5", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "debug_allow_any_pool_priority": "false", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "debug_asok": "1/5", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "debug_asok_assert_abort": "false", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "debug_asserts_on_shutdown": "true", 2026-02-25T08:41:33.587 INFO:teuthology.orchestra.run.trial042.stdout: "debug_auth": "20/20", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_bdev": "1/3", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_bluefs": "1/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_bluestore": "1/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_bluestore_compression": "1/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_buffer": "0/1", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_ceph_dedup": "0/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_ceph_exporter": "1/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_cephfs_mirror": "0/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_cephsqlite": "0/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_client": "0/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_compressor": "1/5", 2026-02-25T08:41:33.588 INFO:teuthology.orchestra.run.trial042.stdout: "debug_context": "0/1", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_crimson_interrupt": "0/5", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_crush": "1/1", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_crypto": "1/5", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_cyanstore": "0/5", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_deliberately_leak_memory": "false", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_disable_randomized_ping": "false", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_dpdk": "1/5", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_eventtrace": "1/5", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_filer": "0/1", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_filestore": "1/3", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_finisher": "1/1", 2026-02-25T08:41:33.589 INFO:teuthology.orchestra.run.trial042.stdout: "debug_fuse": "1/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_heartbeat_testing_span": "0", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_heartbeatmap": "1/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_immutable_obj_cache": "0/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_javaclient": "1/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_journal": "1/3", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_journaler": "0/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_kstore": "1/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_lockdep": "0/1", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mclock": "1/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mds": "1/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mds_balancer": "1/5", 2026-02-25T08:41:33.590 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mds_locker": "1/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mds_log": "1/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mds_log_expire": "1/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mds_migrator": "1/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mds_quiesce": "3/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_memstore": "1/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mgr": "2/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mgrc": "1/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_mon": "20/20", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_monc": "0/10", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_ms": "1/1", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_none": "0/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_objclass": "0/5", 2026-02-25T08:41:33.591 INFO:teuthology.orchestra.run.trial042.stdout: "debug_objectcacher": "0/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_objecter": "0/1", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_optracker": "0/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_osd": "1/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_paxos": "20/20", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_perfcounter": "1/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_prioritycache": "1/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rados": "0/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rbd": "0/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rbd_mirror": "0/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rbd_pwl": "0/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rbd_replay": "0/5", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_refs": "0/0", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_reserver": "1/1", 2026-02-25T08:41:33.592 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_access": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_bucket_logging": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_datacache": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_dbstore": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_dedup": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_flight": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_lifecycle": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_notification": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_restore": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rgw_sync": "1/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_rocksdb": "4/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore": "0/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_backref": "0/5", 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_cache": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_cleaner": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_device": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_epm": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_fixedkv_tree": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_journal": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_lba": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_odata": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_omap": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_onode": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_t": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_seastore_tm": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_striper": "0/1", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_test": "0/5", 2026-02-25T08:41:33.594 INFO:teuthology.orchestra.run.trial042.stdout: "debug_throttle": "1/1", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "debug_timer": "0/1", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "debug_tp": "0/5", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "debug_trace": "1/5", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "defer_client_eviction_on_laggy_osds": "false", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "device_failure_prediction_mode": "none", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "ec_extent_cache_size": "10485760", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "ec_pdw_write_mode": "0", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "enable_availability_tracking": "false", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "crimson", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "err_to_graylog": "false", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "err_to_journald": "false", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "err_to_stderr": "true", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "err_to_syslog": "false", 2026-02-25T08:41:33.595 INFO:teuthology.orchestra.run.trial042.stdout: "event_tracing": "false", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "exporter_addr": "0.0.0.0", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "exporter_cert_file": "", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "exporter_http_port": "9926", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "exporter_key_file": "", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "exporter_prio_limit": "5", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "exporter_sock_dir": "/var/run/ceph/", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "exporter_sort_metrics": "true", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "exporter_stats_period": "5", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "fake_statfs_for_testing": "0", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "fatal_signal_handlers": "true", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "filer_max_purge_ops": "10", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "filer_max_truncate_ops": "128", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_apply_finisher_threads": "1", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_blackhole": "false", 2026-02-25T08:41:33.596 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_btrfs_clone_range": "true", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_btrfs_snap": "true", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_caller_concurrency": "10", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_collect_device_partition_information": "true", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_commit_timeout": "600.000000", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_debug_inject_read_err": "false", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_debug_omap_check": "false", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_debug_verify_split": "false", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_dump_file": "", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_expected_throughput_ops": "200.000000", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_fadvise": "true", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_fail_eio": "true", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_fd_cache_shards": "16", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_fd_cache_size": "128", 2026-02-25T08:41:33.597 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_fiemap": "false", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_fiemap_threshold": "4096", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_fsync_flushes_journal_data": "false", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_index_retry_probability": "0.000000", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_inject_stall": "0", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_journal_parallel": "false", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_journal_trailing": "false", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_journal_writeahead": "false", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_kill_at": "0", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_alloc_hint_size": "1048576", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_inline_xattr_size": "0", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_inline_xattr_size_other": "512", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_inline_xattrs": "0", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2026-02-25T08:41:33.598 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_inline_xattrs_other": "2", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_inline_xattrs_xfs": "10", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_sync_interval": "5.000000", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_xattr_value_size": "0", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_xattr_value_size_other": "1024", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_merge_threshold": "-10", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_min_sync_interval": "0.010000", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_odsync_write": "false", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_omap_backend": "rocksdb", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_omap_backend_path": "", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_omap_header_cache_size": "1024", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_ondisk_finisher_threads": "1", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_op_thread_suicide_timeout": "180", 2026-02-25T08:41:33.599 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_op_thread_timeout": "60", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_op_threads": "2", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_punch_hole": "false", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_high_threshhold": "0.900000", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_low_threshhold": "0.300000", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_max_bytes": "104857600", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_queue_max_ops": "50", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_seek_data_hole": "false", 2026-02-25T08:41:33.600 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_sloppy_crc": "false", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_sloppy_crc_block_size": "65536", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_splice": "false", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_split_multiple": "2", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_split_rand_factor": "20", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_update_to": "1000", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_enable": "true", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2026-02-25T08:41:33.601 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_xfs_extsize": "false", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "filestore_zfs_snap": "false", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fio_dir": "/tmp/fio", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fsid": "517215f2-d42c-4e04-9968-4190617e61fc", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_allow_other": "true", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_atomic_o_trunc": "true", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_big_writes": "true", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_debug": "false", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_default_permissions": "false", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_disable_pagecache": "false", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_max_write": "0", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_multithreaded": "true", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_require_active_mds": "true", 2026-02-25T08:41:33.602 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_set_user_groups": "true", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_splice_move": "true", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_splice_read": "true", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_splice_write": "true", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_syncfs_on_mksnap": "true", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "fuse_use_invalidate_cb": "true", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "gss_target_name": "ceph", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "heartbeat_file": "", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "heartbeat_inject_failure": "0", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "heartbeat_interval": "5", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "host": "trial042", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_max_inflight_ops": "128", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_max_size": "1073741824", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_qos_bps_burst": "0", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2026-02-25T08:41:33.603 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_qos_bps_limit": "0", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_qos_iops_burst": "0", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_qos_iops_limit": "0", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "immutable_object_cache_watermark": "0.900000", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "inject_early_sigterm": "false", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "jaeger_agent_port": "6799", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "jaeger_tracing_enable": "false", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "journal_aio": "true", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "journal_align_min_size": "65536", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "journal_block_align": "true", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "journal_block_size": "4096", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "journal_dio": "true", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "journal_discard": "false", 2026-02-25T08:41:33.604 INFO:teuthology.orchestra.run.trial042.stdout: "journal_force_aio": "false", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_ignore_corruption": "false", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_max_write_bytes": "10485760", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_max_write_entries": "100", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_replay_from": "0", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_throttle_high_multiple": "0.000000", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_throttle_high_threshhold": "0.900000", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_throttle_low_threshhold": "0.600000", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_throttle_max_multiple": "0.000000", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_write_header_frequency": "0", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journal_zero_on_create": "false", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journaler_prefetch_periods": "10", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journaler_prezero_periods": "5", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "journaler_write_head_interval": "15", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "key": "", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "keyfile": "", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "librados_thread_count": "2", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "lockdep": "false", 2026-02-25T08:41:33.605 INFO:teuthology.orchestra.run.trial042.stdout: "lockdep_force_backtrace": "false", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_coarse_timestamps": "true", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_flush_on_exit": "false", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_graylog_host": "127.0.0.1", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_graylog_port": "12201", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_max_new": "1000", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_max_recent": "10000", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_stderr_prefix": "", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_stop_at_utilization": "0.970000", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_to_file": "true", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_to_graylog": "false", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_to_journald": "false", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_to_stderr": "false", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "log_to_syslog": "false", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "max_rotating_auth_attempts": "10", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "mds_action_on_write_error": "1", 2026-02-25T08:41:33.606 INFO:teuthology.orchestra.run.trial042.stdout: "mds_allow_async_dirops": "true", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_allow_batched_ops": "true", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_alternate_name_max": "8192", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_asio_thread_count": "2", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_export_pin": "true", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_fragment_dirs": "true", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_fragment_interval": "5", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_fragment_size_max": "100000", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_idle_threshold": "0.000000", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_interval": "10", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_max": "-1", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_max_until": "-1", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_merge_size": "50", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_midchunk": "0.300000", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_min_rebalance": "0.100000", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_min_start": "0.200000", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_minchunk": "0.001000", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_mode": "0", 2026-02-25T08:41:33.607 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_need_max": "1.200000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_need_min": "0.800000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_overload_epochs": "2", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_replicate_threshold": "8000.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_sample_interval": "3.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_split_bits": "3", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_split_rd": "25000.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_split_size": "10000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_split_wr": "10000.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_target_decay": "10.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_beacon_grace": "15.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_beacon_interval": "4.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_beacon_mon_down_grace": "60", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_memory_limit": "4294967296", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_mid": "0.700000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_quiesce_delay": "0", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_quiesce_sleep": "200", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_quiesce_splitauth": "true", 2026-02-25T08:41:33.608 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_quiesce_threshold": "524288", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_release_free_interval": "10", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_reservation": "0.050000", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_trim_decay_rate": "1.000000", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_trim_interval": "1", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cache_trim_threshold": "262144", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_client_delegate_inos_pct": "50", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_client_prealloc_inos": "1000", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_connect_bootstrapping": "false", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_damage_table_max_entries": "10000", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_debug_auth_pins": "false", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_debug_frag": "false", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_debug_scatterstat": "false", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_debug_subtrees": "false", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_decay_halflife": "5.000000", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_default_dir_hash": "2", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_defer_session_stale": "true", 2026-02-25T08:41:33.609 INFO:teuthology.orchestra.run.trial042.stdout: "mds_delay_journal_replay_for_testing": "0", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_deny_all_reconnect": "false", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dir_keys_per_op": "16384", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dir_max_commit_size": "10", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dir_max_entries": "0", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dir_prefetch": "true", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dirstat_min_interval": "1.000000", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dump_cache_after_rejoin": "false", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dump_cache_on_map": "false", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dump_cache_threshold_file": "0", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_early_reply": "true", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_enable_op_tracker": "true", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_enforce_unique_name": "true", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_export_ephemeral_distributed": "true", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_export_ephemeral_random": "true", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_export_ephemeral_random_max": "0.010000", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_extraordinary_events_dump_interval": "0", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_file_blockdiff_max_concurrent_object_scans": "16", 2026-02-25T08:41:33.610 INFO:teuthology.orchestra.run.trial042.stdout: "mds_forward_all_requests_to_auth": "false", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_freeze_tree_timeout": "30.000000", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_fscrypt_last_block_max_size": "4096", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_go_bad_corrupt_dentry": "true", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_health_cache_threshold": "1.500000", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_health_summarize_threshold": "10", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_heartbeat_grace": "15.000000", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_heartbeat_reset_grace": "1000", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_inject_health_dummy": "false", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_inject_migrator_session_race": "false", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_inject_skip_replaying_inotable": "false", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_join_fs": "", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_journal_format": "1", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_after_journal_logs_flushed": "false", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_create_at": "0", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_dirfrag_at": "0", 2026-02-25T08:41:33.611 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_export_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_import_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_journal_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_journal_expire_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_journal_replay_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_link_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_mdstable_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_openc_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_rename_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_kill_shutdown_at": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_event_large_threshold": "524288", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_events_per_segment": "1024", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_max_events": "-1", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_max_segments": "128", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_minor_segments_per_major_segment": "16", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_pause": "false", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_segment_size": "0", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_skip_corrupt_events": "false", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_skip_unbounded_events": "false", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_trim_decay_rate": "1.000000", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_trim_threshold": "128", 2026-02-25T08:41:33.612 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_trim_upkeep_interval": "1000", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_log_warn_factor": "2.000000", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_caps_per_client": "1048576", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_completed_flushes": "100000", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_completed_requests": "100000", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_export_size": "20971520", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_file_recover": "32", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_purge_files": "64", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_purge_ops": "8192", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_scrub_ops_in_progress": "5", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_max_snaps_per_dir": "100", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_metrics_update_interval": "2", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_min_caps_per_client": "100", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_min_caps_working_set": "10000", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_mon_shutdown_timeout": "5.000000", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_numa_node": "-1", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_oft_prefetch_dirfrags": "false", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_op_complaint_time": "30.000000", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_op_history_duration": "600", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_op_history_size": "20", 2026-02-25T08:41:33.613 INFO:teuthology.orchestra.run.trial042.stdout: "mds_op_history_slow_op_size": "20", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_op_history_slow_op_threshold": "10", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_op_log_threshold": "5", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_ping_grace": "15", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_ping_interval": "5", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_recall_global_max_decay_threshold": "131072", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_recall_max_caps": "30000", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_recall_max_decay_rate": "1.500000", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_recall_max_decay_threshold": "131072", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_recall_warning_decay_rate": "60.000000", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_recall_warning_threshold": "262144", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_reconnect_timeout": "45.000000", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_replay_interval": "1.000000", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_replay_unsafe_with_closed_session": "false", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_request_load_average_decay_rate": "60.000000", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_root_ino_gid": "0", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_root_ino_uid": "0", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_scatter_nudge_interval": "5.000000", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_scrub_stats_review_period": "1", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_server_dispatch_client_request_delay": "0", 2026-02-25T08:41:33.614 INFO:teuthology.orchestra.run.trial042.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_session_blocklist_on_evict": "true", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_session_blocklist_on_timeout": "true", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_session_cache_liveness_magnitude": "10", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_session_cap_acquisition_throttle": "100000", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_session_metadata_threshold": "16777216", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_sessionmap_keys_per_op": "1024", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_shutdown_check": "0", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_skip_ino": "0", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_sleep_rank_change": "0.000000", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_snap_max_uid": "4294967294", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_snap_min_uid": "0", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_snap_rstat": "false", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_standby_replay_damaged": "false", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_symlink_recovery": "true", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_task_status_update_interval": "2.000000", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_thrash_exports": "0", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_thrash_fragments": "0", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_tick_interval": "5.000000", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_use_global_snaprealm_seq_for_subvol": "true", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_valgrind_exit": "false", 2026-02-25T08:41:33.615 INFO:teuthology.orchestra.run.trial042.stdout: "mds_verify_backtrace": "1", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mds_verify_scatter": "false", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mds_wipe_ino_prealloc": "false", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mds_wipe_sessions": "false", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mempool_debug": "false", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "memstore_debug_omit_block_device_write": "false", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "memstore_device_bytes": "1073741824", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "memstore_page_set": "false", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "memstore_page_size": "65536", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_client_bytes": "134217728", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_client_messages": "512", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_connect_retry_interval": "1.000000", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_disabled_modules": "", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_enable_op_tracker": "true", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_initial_modules": "iostat nfs", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_max_pg_creating": "1024", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_max_pg_num_change": "128", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_mds_bytes": "134217728", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_mds_messages": "128", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2026-02-25T08:41:33.616 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_mon_bytes": "134217728", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_mon_messages": "128", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_num_op_tracker_shard": "32", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_op_complaint_time": "30.000000", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_op_history_duration": "600", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_op_history_size": "20", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_op_history_slow_op_size": "20", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_op_log_threshold": "5", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_osd_bytes": "536870912", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_osd_messages": "8192", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_pool": "true", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_service_beacon_grace": "60.000000", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_standby_modules": "true", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_stats_period": "5", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_stats_threshold": "5", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_tick_period": "2", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mgr_ttl_cache_expire_seconds": "0", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mon_accept_timeout_factor": "2.000000", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mon_allow_pool_delete": "true", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mon_allow_pool_size_one": "true", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mon_auth_validate_all_caps": "true", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2026-02-25T08:41:33.617 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_bytes": "104857600", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_directed_command_retry": "2", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_hunt_interval": "3.000000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_hunt_on_resend": "true", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_hunt_parallel": "3", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_log_interval": "1.000000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_max_log_entries_per_message": "1000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_ping_interval": "10.000000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_ping_timeout": "30.000000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_client_target_rank": "-1", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_clock_drift_allowed": "1.000000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_level": "debug", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_to_file": "true", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_to_graylog": "false", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_to_graylog_port": "12201", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_to_journald": "false", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_to_stderr": "false", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_to_syslog": "default=false", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2026-02-25T08:41:33.618 INFO:teuthology.orchestra.run.trial042.stdout: "mon_compact_on_bootstrap": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_compact_on_start": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_compact_on_trim": "true", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_con_tracker_persist_interval": "10", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_con_tracker_score_halflife": "43200", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_config_key_max_entry_size": "65536", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_cpu_threads": "4", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_crush_min_required_version": "hammer", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_daemon_bytes": "419430400", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_data_avail_crit": "5", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_data_avail_warn": "5", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_data_size_warn": "16106127360", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_block_osdmap_trim": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_deprecated_as_obsolete": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_dump_json": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_dump_transactions": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_extra_checks": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_no_initial_persistent_features": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_no_require_squid": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_no_require_tentacle": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_delta_reset_interval": "10.000000", 2026-02-25T08:41:33.619 INFO:teuthology.orchestra.run.trial042.stdout: "mon_dns_srv_name": "ceph-mon", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_down_added_grace": "180", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_down_mkfs_grace": "120", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_down_uptime_grace": "60", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_election_default_strategy": "1", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_election_timeout": "5.000000", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_elector_ping_divisor": "2", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_elector_ping_timeout": "2.000000", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_enable_op_tracker": "true", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_fake_pool_delete": "false", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_force_quorum_join": "false", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_fsmap_prune_threshold": "300", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_globalid_prealloc": "10000", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_health_detail_to_clog": "false", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_health_log_update_period": "5", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_health_max_detail": "50", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_health_to_clog": "true", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_health_to_clog_interval": "600", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_host": "10.20.193.42", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_host_override": "", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_initial_members": "", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_inject_transaction_delay_max": "10.000000", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_keyvaluedb": "rocksdb", 2026-02-25T08:41:33.620 INFO:teuthology.orchestra.run.trial042.stdout: "mon_lease": "5.000000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_lease_renew_interval_factor": "0.600000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_log_full_interval": "50", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_log_max": "10000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_log_max_summary": "50", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_log_entries_per_event": "4096", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_log_epochs": "500", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_mdsmap_epochs": "500", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_mgrmap_epochs": "500", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_nvmeof_epochs": "500", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_osd": "10000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_pg_per_osd": "10000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_pool_pg_num": "65536", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_max_snap_prune_per_epoch": "100", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mds_blocklist_interval": "86400.000000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mds_force_trim_to": "0", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mds_skip_sanity": "false", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_memory_autotune": "true", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_memory_target": "2147483648", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mgr_beacon_grace": "30", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mgr_digest_period": "5", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mgr_inactive_grace": "60", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mgr_mkfs_grace": "240", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_min_osdmap_epochs": "50", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_netsplit_grace_period": "9", 2026-02-25T08:41:33.621 INFO:teuthology.orchestra.run.trial042.stdout: "mon_nvmeofgw_beacon_grace": "7", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_nvmeofgw_beacons_till_ack": "15", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_nvmeofgw_delete_grace": "900", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_nvmeofgw_skip_failovers_interval": "16", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_nvmeofgw_wrong_map_ignore_sec": "15", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_op_complaint_time": "30", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_op_history_duration": "600", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_op_history_size": "20", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_op_history_slow_op_size": "20", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_op_history_slow_op_threshold": "10", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_op_log_threshold": "5", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_adjust_down_out_interval": "true", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_auto_mark_in": "false", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_auto_mark_new_in": "true", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_cache_size": "500", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_cache_size_min": "134217728", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_crush_smoke_test": "true", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_destroyed_out_interval": "600", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_down_out_interval": "600", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_down_out_subtree_limit": "rack", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2026-02-25T08:41:33.622 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_force_trim_to": "0", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_full_ratio": "0.950000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_laggy_halflife": "3600", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_laggy_max_interval": "300", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_laggy_weight": "0.300000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_max_initial_pgs": "1024", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_min_down_reporters": "2", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_min_in_ratio": "0.750000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_min_up_ratio": "0.300000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_nearfull_ratio": "0.850000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_prime_pg_temp": "true", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_report_timeout": "900", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_reporter_subtree_level": "osd", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_warn_num_repaired": "10", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osd_warn_op_age": "32.000000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osdmap_full_prune_enabled": "true", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osdmap_full_prune_interval": "2", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osdmap_full_prune_min": "15", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_osdmap_full_prune_txsize": "2", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_pg_stuck_threshold": "60", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_pg_warn_min_objects": "10000", 2026-02-25T08:41:33.623 INFO:teuthology.orchestra.run.trial042.stdout: "mon_pg_warn_min_per_osd": "0", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_pg_warn_min_pool_objects": "1000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_pool_quota_crit_threshold": "0", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_pool_quota_warn_threshold": "0", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_probe_timeout": "2.000000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_reweight_max_change": "0.050000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_reweight_max_osds": "4", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_reweight_min_bytes_per_osd": "10", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_reweight_min_pgs_per_osd": "4", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_scrub_interval": "86400", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_scrub_max_keys": "100", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_scrub_timeout": "300", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_session_timeout": "300", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_smart_report_timeout": "5", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_stat_smooth_intervals": "6", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_stretch_pool_min_size": "2", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_stretch_pool_size": "4", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_subscribe_interval": "86400.000000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_sync_debug": "false", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_sync_max_payload_keys": "2000", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_sync_max_payload_size": "1048576", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_sync_provider_kill_at": "0", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_sync_requester_kill_at": "0", 2026-02-25T08:41:33.624 INFO:teuthology.orchestra.run.trial042.stdout: "mon_sync_timeout": "60.000000", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_target_pg_per_osd": "100", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_tick_interval": "5", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_timecheck_interval": "300.000000", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_timecheck_skew_interval": "30.000000", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_use_min_delay_socket": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_older_version_delay": "604800", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_filestore_osds": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_misplaced": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_older_version": "true", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_pool_no_app": "true", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_pool_no_app_grace": "300", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_pool_no_redundancy": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_on_too_few_osds": "false", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "monmap": "", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2026-02-25T08:41:33.625 INFO:teuthology.orchestra.run.trial042.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "motr_tracing_enabled": "false", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_op_threads": "3", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_buffer_size": "131072", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_cm": "false", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_device_name": "", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_dscp": "96", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_enable_hugepage": "false", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_gid_idx": "0", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_local_gid": "", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_polling_us": "1000", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_port_num": "1", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_receive_buffers": "32768", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_receive_queue_len": "4096", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_roce_ver": "1", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_send_buffers": "1024", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_sl": "3", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_support_srq": "true", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_rdma_type": "ib", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_async_reap_threshold": "5", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_before_connect": "false", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_ipv4": "true", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_ipv6": "false", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_msgr1": "true", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_msgr2": "true", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_port_max": "7568", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_port_min": "6800", 2026-02-25T08:41:33.626 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_prefer_ipv4": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_retry_count": "3", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_bind_retry_delay": "5", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_blackhole_client": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_blackhole_mds": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_blackhole_mgr": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_blackhole_mon": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_blackhole_osd": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_client_mode": "crc", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_client_throttle_retry_time_interval": "5000", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_cluster_mode": "crc", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_cluster_type": "", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_compress_secure": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_connection_idle_timeout": "900", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_connection_ready_timeout": "10", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_crc_data": "true", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_crc_header": "true", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_die_on_bad_msg": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_die_on_bug": "true", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_die_on_old_message": "true", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_die_on_skipped_message": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_die_on_unhandled_msg": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dispatch_throttle_bytes": "104857600", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_coremask": "0xF", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_debug_allow_loopback": "false", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_devs_allowlist": "", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_enable_tso": "true", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_host_ipv4_addr": "", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_hugepages": "", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_hw_flow_control": "true", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2026-02-25T08:41:33.627 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_lro": "true", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_memory_channel": "4", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_pmd": "", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_port_id": "0", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dump_corrupt_message_level": "1", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_dump_on_send": "false", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_initial_backoff": "0.200000", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_inject_delay_max": "1.000000", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_inject_delay_probability": "0.000000", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_inject_delay_type": "", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_inject_internal_delays": "0.000000", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_inject_network_congestion": "0", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_inject_socket_failures": "0", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_learn_addr_from_peer": "true", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_max_accept_failures": "4", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_max_backoff": "15.000000", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_mon_client_mode": "crc", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_mon_cluster_mode": "crc", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_mon_service_mode": "crc", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_osd_compress_min_size": "1024", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_osd_compress_mode": "none", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_osd_compression_algorithm": "snappy", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_pq_min_cost": "65536", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_public_type": "", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_service_mode": "crc", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_tcp_listen_backlog": "512", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_tcp_nodelay": "true", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_tcp_prefetch_max_size": "65536", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_tcp_rcvbuf": "0", 2026-02-25T08:41:33.628 INFO:teuthology.orchestra.run.trial042.stdout: "ms_time_events_min_wait_interval": "1000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "ms_type": "async+posix", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "no_config_file": "false", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "nvmeof_mon_client_connect_panic": "30", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "nvmeof_mon_client_tick_period": "1", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objecter_completion_locks_per_session": "32", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objecter_debug_inject_relock_delay": "false", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objecter_inflight_op_bytes": "104857600", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objecter_inflight_ops": "1024", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objecter_inject_no_watch_ping": "false", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objecter_retry_writes_after_first_reply": "false", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objecter_tick_interval": "5.000000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objecter_timeout": "10.000000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objectstore_blackhole": "false", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "openssl_conf": "", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_agent_delay_time": "5.000000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_agent_hist_halflife": "1000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_agent_max_low_ops": "2", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_agent_max_ops": "4", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_agent_min_evict_effort": "0.100000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_agent_quantize_effort": "0.100000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_agent_slop": "0.020000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_aggregated_slow_ops_logging": "true", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_allow_recovery_below_min_size": "true", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_asio_thread_count": "2", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_async_recovery_min_cost": "100", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_auto_mark_unfound_lost": "false", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_backfill_retry_interval": "30.000000", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_backfill_scan_max": "512", 2026-02-25T08:41:33.629 INFO:teuthology.orchestra.run.trial042.stdout: "osd_backfill_scan_min": "64", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_backoff_on_degraded": "false", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_backoff_on_peering": "false", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_backoff_on_unfound": "true", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_beacon_report_interval": "300", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_bench_duration": "30", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_bench_large_size_max_throughput": "104857600", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_bench_max_block_size": "67108864", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_bench_small_size_max_iops": "100", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_blkin_trace_all": "false", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_blocked_scrub_grace_period": "120", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_check_for_log_corruption": "false", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_check_max_object_name_len_on_startup": "true", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo sem_set", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo sem_set", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_class_update_on_start": "true", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_client_message_cap": "256", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_client_message_size_cap": "524288000", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_client_op_priority": "63", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_client_watch_timeout": "30", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_command_max_records": "256", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_compact_on_start": "false", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_copyfrom_max_chunk": "8388608", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_crush_chooseleaf_type": "0", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_crush_initial_weight": "-1.000000", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_crush_update_on_start": "true", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_crush_update_weight_set": "true", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2026-02-25T08:41:33.630 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_drop_ping_duration": "0", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_drop_ping_probability": "0.000000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_feed_pullee": "-1", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_inject_copyfrom_error": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_misdirected_ops": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_no_acting_change": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_no_purge_strays": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_op_order": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_pg_log_writeout": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_pretend_recovery_active": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_random_push_read_error": "0.000000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_shutdown": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_trim_objects": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_verify_cached_snaps": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_verify_missing_on_start": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_verify_snaps": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_debug_verify_stray_on_activate": "false", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_deep_scrub_interval": "604800.000000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_deep_scrub_interval_cv": "0.200000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_deep_scrub_keys": "1024", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_deep_scrub_stride": "4194304", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_default_data_pool_replay_window": "5", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_default_notify_timeout": "30", 2026-02-25T08:41:33.631 INFO:teuthology.orchestra.run.trial042.stdout: "osd_delete_sleep": "0.000000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_delete_sleep_hdd": "5.000000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_delete_sleep_hybrid": "1.000000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_delete_sleep_ssd": "1.000000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_discard_disconnected_ops": "true", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ec_partial_reads": "true", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_enable_op_tracker": "true", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_extblkdev_plugins": "vdo", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_failsafe_full_ratio": "0.970000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_fast_fail_on_connection_refused": "true", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_fast_info": "true", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_fast_shutdown": "true", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_fast_shutdown_notify_mon": "true", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_fast_shutdown_timeout": "15", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_find_best_info_ignore_history_les": "false", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_force_auth_primary_missing_objects": "100", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_function_tracing": "false", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_heartbeat_grace": "20", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_heartbeat_interval": "6", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_heartbeat_min_peers": "10", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_heartbeat_min_size": "2000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_heartbeat_stale": "600", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_hit_set_max_size": "100000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_hit_set_min_size": "1000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_hit_set_namespace": ".ceph-internal", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ignore_stale_divergent_priors": "false", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_inject_failure_on_pg_removal": "false", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2026-02-25T08:41:33.632 INFO:teuthology.orchestra.run.trial042.stdout: "osd_journal_flush_on_shutdown": "true", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_journal_size": "5120", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_kill_backfill_at": "0", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_loop_before_reset_tphandle": "64", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_map_cache_size": "50", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_map_dedup": "true", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_map_message_max": "40", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_map_message_max_bytes": "10485760", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_map_share_max_epochs": "40", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_attr_name_len": "100", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_attr_size": "0", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_backfills": "1", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_markdown_count": "5", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_markdown_period": "600", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_object_name_len": "2048", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_object_namespace_len": "256", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_object_size": "134217728", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_omap_entries_per_request": "1024", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_pg_blocked_by": "16", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_pg_log_entries": "10000", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_pgls": "1024", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_push_cost": "8388608", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_push_objects": "10", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_scrubs": "3", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_trimming_pgs": "2", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_write_op_reply_len": "64", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_max_write_size": "90", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_iops_capacity_low_threshold_hdd": "50.000000", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_iops_capacity_low_threshold_ssd": "1000.000000", 2026-02-25T08:41:33.633 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_override_recovery_settings": "false", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_profile": "balanced", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_scheduler_client_wgt": "1", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mclock_skip_benchmark": "false", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_memory_base": "805306368", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_memory_cache_min": "134217728", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_memory_cache_resize_interval": "1.000000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_memory_expected_fragmentation": "0.150000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_memory_target": "4294967296", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_memory_target_autotune": "false", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_min_pg_log_entries": "250", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_min_recovery_priority": "0", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mon_heartbeat_interval": "30", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mon_report_interval": "5", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mon_report_max_in_flight": "2", 2026-02-25T08:41:33.634 INFO:teuthology.orchestra.run.trial042.stdout: "osd_mon_shutdown_timeout": "5.000000", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_num_cache_shards": "32", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_num_op_tracker_shard": "32", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_numa_auto_affinity": "true", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_numa_node": "-1", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_numa_prefer_iface": "true", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_object_clean_region_max_num_intervals": "10", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_objecter_finishers": "1", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_objectstore": "bluestore", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_objectstore_fuse": "false", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_objectstore_ideal_list_max": "64", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_objectstore_tracing": "false", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_complaint_time": "30.000000", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_history_duration": "600", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_history_size": "20", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_history_slow_op_size": "20", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_log_threshold": "5", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_num_shards": "0", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_num_shards_hdd": "1", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_num_shards_ssd": "8", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_num_threads_per_shard": "0", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_num_threads_per_shard_hdd": "5", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_pq_min_cost": "65536", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_queue": "mclock_scheduler", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_queue_cut_off": "high", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_thread_suicide_timeout": "150", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_op_thread_timeout": "15", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_open_classes_on_start": "true", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_os_flags": "0", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_peering_op_priority": "255", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_delete_cost": "1048576", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_delete_priority": "5", 2026-02-25T08:41:33.635 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_log_dups_tracked": "3000", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_log_trim_max": "10000", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_log_trim_min": "100", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_object_context_cache_count": "64", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_cache_min_evict_age": "0", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_cache_min_flush_age": "0", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_crimson": "true", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_crush_rule": "-1", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_ec_fast_read": "false", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_erasure_code_profile": "plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_flag_bulk": "false", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_flag_ec_optimizations": "false", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_flag_hashpspool": "true", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_flag_nodelete": "false", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_flag_nopgchange": "false", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_flag_nosizechange": "false", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_flags": "0", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_min_size": "2", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_pg_num": "32", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_pgp_num": "0", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_read_ratio": "70", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_size": "3", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_default_type": "replicated", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_erasure_code_stripe_unit": "0", 2026-02-25T08:41:33.636 INFO:teuthology.orchestra.run.trial042.stdout: "osd_pool_use_gmt_hitset": "true", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_push_per_object_cost": "1000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_read_ec_check_for_errors": "false", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recover_clone_overlap": "true", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recover_clone_overlap_limit": "10", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_cost": "20971520", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_delay_start": "0.000000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_max_active": "0", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_max_active_hdd": "3", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_max_active_ssd": "10", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_max_chunk": "8388608", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_max_single_start": "1", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_op_priority": "3", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_op_warn_multiple": "16", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_priority": "5", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_retry_interval": "30.000000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_sleep": "0.000000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_sleep_degraded": "0.000000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_sleep_degraded_hdd": "0.100000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_sleep_degraded_hybrid": "0.025000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_sleep_degraded_ssd": "0.000000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_sleep_hdd": "0.100000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_recovery_sleep_ssd": "0.000000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_requested_scrub_priority": "5", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_rollback_to_cluster_snap": "", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_auto_repair": "false", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_auto_repair_num_errors": "5", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_backoff_ratio": "0.660000", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_begin_hour": "0", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_begin_week_day": "0", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_chunk_max": "15", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_chunk_min": "5", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_cost": "52428800", 2026-02-25T08:41:33.637 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_disable_reservation_queuing": "false", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_during_recovery": "false", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_end_hour": "0", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_end_week_day": "0", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_event_cost": "4096", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_extended_sleep": "0.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_invalid_stats": "true", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_load_threshold": "10.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_max_interval": "604800.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_max_preemptions": "5", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_min_interval": "86400.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_priority": "5", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_retry_after_noscrub": "60", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_retry_delay": "30", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_retry_new_interval": "10", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_retry_pg_state": "60", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_retry_trimming": "10", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_scrub_sleep": "0.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_shallow_scrub_chunk_max": "100", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_shallow_scrub_chunk_min": "50", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_shutdown_pgref_assert": "false", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_skip_check_past_interval_bounds": "false", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_skip_data_digest": "false", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_smart_report_timeout": "5", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_snap_trim_cost": "1048576", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_snap_trim_priority": "5", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_snap_trim_sleep": "0.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_stats_update_period_not_scrubbing": "120", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_stats_update_period_scrubbing": "15", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_target_transaction_size": "30", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_default_cache_hit_set_count": "4", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2026-02-25T08:41:33.638 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_default_cache_mode": "writeback", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tier_promote_max_objects_sec": "25", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_tracing": "false", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_use_stale_snap": "false", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "osdc_blkin_trace_all": "false", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_kill_at": "0", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_max_join_drift": "10", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_min": "500", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_min_wait": "0.050000", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_propose_interval": "1.000000", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_service_trim_max": "500", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_service_trim_max_multiplier": "20", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_service_trim_min": "10", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_stash_full_interval": "25", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_trim_max": "500", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "paxos_trim_min": "250", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "perf": "true", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "plugin_crypto_accelerator": "crypto_isal", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "plugin_dir": "/usr/lib64/ceph", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "pool_availability_update_interval": "1.000000", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "public_addr": "-", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "public_addrv": "", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "public_bind_addr": "-", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "public_network": "", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "public_network_interface": "", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "qat_compressor_busy_polling": "false", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "qat_compressor_enabled": "false", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "qat_compressor_session_max_number": "256", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "rados_mon_op_timeout": "0", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "rados_osd_op_timeout": "0", 2026-02-25T08:41:33.639 INFO:teuthology.orchestra.run.trial042.stdout: "rados_replica_read_policy": "default", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rados_replica_read_policy_on_objclass": "false", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rados_tracing": "false", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_atime_update_interval": "60", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_balance_parent_reads": "false", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_balance_snap_reads": "false", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_blkin_trace_all": "false", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_blocklist_expire_seconds": "0", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_blocklist_on_break_lock": "true", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache": "true", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache_block_writes_upfront": "false", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache_max_dirty": "25165824", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache_max_dirty_age": "1.000000", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache_max_dirty_object": "0", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache_policy": "writearound", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache_size": "33554432", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache_target_dirty": "16777216", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_cache_writethrough_until_flush": "true", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_clone_copy_on_read": "false", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_compression_hint": "none", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_concurrent_management_ops": "10", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_config_pool_override_update_timestamp": "0", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_clone_format": "auto", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_data_pool": "", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_features": "61", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_format": "2", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_map_options": "", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_order": "22", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_pool": "rbd", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_stripe_count": "0", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_default_stripe_unit": "0", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_disable_zero_copy_writes": "true", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_discard_granularity_bytes": "65536", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_discard_on_zeroed_write_same": "true", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_enable_alloc_hint": "true", 2026-02-25T08:41:33.640 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_io_scheduler": "simple", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_commit_age": "5.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_max_payload_bytes": "16384", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_object_flush_age": "0.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_object_flush_bytes": "1048576", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_object_flush_interval": "0", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_order": "24", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_pool": "", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_journal_splay_width": "4", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_localize_parent_reads": "false", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_localize_snap_reads": "false", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_image_policy_type": "simple", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_image_state_check_interval": "30", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_memory_autotune": "true", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_memory_base": "805306368", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_memory_cache_min": "134217728", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_memory_target": "4294967296", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_perf_stats_prio": "5", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2026-02-25T08:41:33.641 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirroring_delete_delay": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirroring_replay_delay": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_move_to_trash_on_remove": "false", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_mtime_update_interval": "60", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_non_blocking_aio": "true", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_op_thread_timeout": "60", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_op_threads": "1", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_parent_cache_enabled": "false", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_persistent_cache_mode": "disabled", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_persistent_cache_path": "/tmp", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_persistent_cache_size": "1073741824", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_plugins": "", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_bps_burst": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_bps_burst_seconds": "1", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_bps_limit": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_exclude_ops": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_iops_burst": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_iops_burst_seconds": "1", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_iops_limit": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_read_bps_burst": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_read_bps_limit": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_read_iops_burst": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_read_iops_limit": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_schedule_tick_min": "50", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_write_bps_burst": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_write_bps_limit": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_write_iops_burst": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_qos_write_iops_limit": "0", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_quiesce_notification_attempts": "10", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_read_from_replica_policy": "default", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2026-02-25T08:41:33.642 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_readahead_max_bytes": "524288", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_readahead_trigger_requests": "10", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_request_timed_out_seconds": "30", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_skip_partial_discard": "true", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_tracing": "false", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_validate_names": "true", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rbd_validate_pool": "true", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "restapi_base_url": "", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "restapi_log_level": "", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_account_default_quota_max_objects": "-1", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_account_default_quota_max_size": "-1", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_acl_grants_max_num": "100", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_admin_entry": "admin", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_asio_assert_yielding": "false", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_backend_store": "rados", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_barbican_url": "", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_beast_enable_async": "true", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_counters_cache": "false", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_counters_cache_size": "10000", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_default_quota_max_size": "-1", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_eexist_override": "false", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_index_max_aio": "128", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_logging_obj_roll_time": "300", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_persistent_notif_num_shards": "11", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_quota_cache_size": "10000", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_quota_ttl": "600", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_bucket_sync_spawn_window": "20", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_cache_enabled": "true", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_cache_expiry_interval": "900", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_cache_lru_size": "25000", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_config_store": "rados", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_content_length_compat": "false", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_copy_obj_progress": "true", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_cors_rules_max_num": "100", 2026-02-25T08:41:33.643 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_cross_domain_policy": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_default_encryption_key": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_kmip_addr": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_kmip_ca_path": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_kmip_client_cert": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_kmip_client_key": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_kmip_kms_key_template": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_kmip_password": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_kmip_username": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_require_ssl": "true", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_backend": "vault", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_suppress_logs": "true", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_addr": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_auth": "token", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_namespace": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_prefix": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_secret_engine": "transit", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_ssl_cacert": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_token_file": "", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_crypt_vault_verify_ssl": "true", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_curl_buffersize": "524288", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_curl_low_speed_limit": "1024", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_curl_low_speed_time": "300", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_curl_tcp_keepalive": "0", 2026-02-25T08:41:33.644 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_curl_wait_timeout_ms": "1000", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d3n_l1_fadvise": "4", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d3n_libaio_aio_num": "64", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d3n_libaio_aio_threads": "20", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_backend_address": "127.0.0.1:6379", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_cache_cleaning_interval": "1000", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_l1_datacache_disk_reserve": "1073741824", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_l1_fadvise": "4", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_l1_write_open_flags": "4096", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_libaio_aio_num": "64", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_libaio_aio_threads": "20", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_d4n_local_rgw_address": "127.0.0.1:8000", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_data_log_changes_size": "1000", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_data_log_num_shards": "128", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_data_log_window": "30", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_data_notify_interval_msec": "0", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_data_sync_poll_interval": "20", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_data_sync_spawn_window": "20", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_debug_inject_set_olh_err": "0", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_default_data_log_backing": "fifo", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_default_realm_info_oid": "default.realm", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_default_region_info_oid": "default.region", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_default_zone_info_oid": "default.zone", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_defer_to_bucket_acls": "", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_delete_multi_obj_max_num": "1000", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_disable_s3select": "false", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_admin_lim": "0.000000", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_admin_res": "100.000000", 2026-02-25T08:41:33.645 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_auth_lim": "0.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_auth_res": "200.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_data_lim": "0.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_data_res": "500.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_data_wgt": "500.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_metadata_res": "500.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dns_name": "", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dns_s3website_name": "", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dynamic_resharding": "true", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_gc_threads": "true", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_jwks_url_verification": "false", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_lc_threads": "true", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_mdsearch": "true", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_ops_log": "false", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_quota_threads": "true", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_restore_threads": "true", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_static_website": "false", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enable_usage_log": "false", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_enforce_swift_acls": "true", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_exit_timeout_secs": "120", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_expose_bucket": "false", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_extended_http_attrs": "", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_filter": "none", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_frontends": "beast port=7480", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_max_concurrent_io": "10", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_max_deferred": "50", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_max_objs": "32", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_max_queue_size": "134213632", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_max_trim_chunk": "16", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_obj_min_wait": "7200", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_processor_max_time": "3600", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_gc_processor_period": "3600", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_get_obj_max_req_size": "4194304", 2026-02-25T08:41:33.646 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_get_obj_window_size": "16777216", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_graceful_stop": "false", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_healthcheck_disabling_path": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_http_notif_connection_timeout": "5", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_http_notif_max_inflight": "8192", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_http_notif_message_timeout": "10", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ignore_get_invalid_range": "false", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_init_timeout": "300", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_kafka_connection_idle": "300", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_kafka_message_timeout": "5000", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_kafka_sleep_timeout": "10", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_accepted_admin_roles": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_accepted_reader_roles": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_admin_domain": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_admin_password": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_admin_password_path": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_admin_project": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_admin_tenant": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_admin_user": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_barbican_domain": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_barbican_password": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_barbican_project": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_barbican_tenant": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_barbican_user": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_implicit_tenants": "false", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_service_token_enabled": "false", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_token_cache_size": "10000", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_token_cache_ttl": "300", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_url": "", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_keystone_verify_ssl": "true", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_debug_interval": "-1", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_list_cnt": "1000", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_lock_max_time": "90", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_max_objs": "32", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_max_rules": "1000", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_max_worker": "3", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_max_wp_worker": "128", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_ordered_list_threshold": "500", 2026-02-25T08:41:33.647 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lc_thread_delay": "0", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ldap_dnattr": "uid", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ldap_searchfilter": "", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ldap_uri": "", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lfuda_sync_frequency": "60", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_list_bucket_min_readahead": "1000", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_list_buckets_max_chunk": "1000", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_log_http_headers": "", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_log_nonexistent_bucket": "false", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_log_object_name_utc": "false", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lua_enable": "true", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lua_max_memory_per_state": "128000", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_lua_max_runtime_per_state": "1000", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_attr_name_len": "0", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_attr_size": "0", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_attrs_num_in_req": "0", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_chunk_size": "4194304", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_concurrent_requests": "1024", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_control_aio": "8", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_dynamic_shards": "1999", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_listing_results": "5000", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_notify_retries": "10", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_objs_per_shard": "100000", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_put_param_size": "1048576", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_put_size": "5368709120", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_max_slo_entries": "1000", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_md_log_max_shards": "64", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_md_notify_interval_msec": "200", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_meta_sync_poll_interval": "20", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_meta_sync_spawn_window": "20", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_mime_types_file": "/etc/mime.types", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_mp_lock_max_time": "600", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_multi_obj_del_max_aio": "16", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_multipart_min_part_size": "5242880", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_multipart_part_upload_limit": "10000", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_fhcache_partitions": "3", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_fhcache_size": "2017", 2026-02-25T08:41:33.648 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_frontends": "rgw-nfs", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_lru_lanes": "5", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_max_gc": "300", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_namespace_expire_secs": "300", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_run_gc_threads": "false", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_run_lc_threads": "false", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_run_quota_threads": "false", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_run_restore_threads": "false", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_run_sync_thread": "false", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_s3_fast_attrs": "false", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_nfs_write_completion_interval_s": "10", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_num_async_rados_threads": "32", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_num_control_oids": "8", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_numa_node": "-1", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_obj_stripe_size": "4194304", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_obj_tombstone_cache_size": "1000", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_objexp_chunk_size": "100", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_objexp_gc_interval": "600", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_objexp_hints_num_shards": "127", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_olh_pending_timeout_sec": "3600", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_op_thread_suicide_timeout": "0", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_op_thread_timeout": "600", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_op_tracing": "false", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_opa_token": "", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_opa_url": "", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_opa_verify_ssl": "true", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ops_log_data_backlog": "5242880", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ops_log_rados": "false", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ops_log_socket_path": "", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_override_bucket_index_max_shards": "0", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_parquet_buffer_size": "16777216", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_period_push_interval": "2.000000", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_period_push_interval_max": "30.000000", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_period_root_pool": ".rgw.root", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_policy_reject_invalid_principals": "true", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_posix_cache_lanes": "3", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_posix_cache_lmdb_count": "3", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_posix_cache_max_buckets": "100", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_posix_cache_partitions": "3", 2026-02-25T08:41:33.649 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_posix_userdb_dir": "/var/lib/ceph/radosgw", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_print_continue": "true", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_print_prohibited_content_length": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_put_obj_max_window_size": "67108864", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_put_obj_min_window_size": "16777216", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_rados_pool_recovery_priority": "5", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_rados_tracing": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_ratelimit_interval": "60", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_realm": "", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_realm_id": "", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_realm_root_pool": ".rgw.root", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_redis_connection_pool_size": "512", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_region": "", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_region_root_pool": ".rgw.root", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_relaxed_region_enforcement": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_relaxed_s3_bucket_names": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_request_uri": "", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshard_batch_size": "64", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshard_bucket_lock_duration": "360", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshard_debug_interval": "-1", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshard_max_aio": "128", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshard_num_logs": "16", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshard_progress_judge_interval": "120", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshard_thread_interval": "600", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_reshardlog_threshold": "30000", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_resolve_cname": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_rest_getusage_op_compat": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_restore_debug_interval": "-1", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_restore_lock_max_time": "90", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_restore_max_objs": "32", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_restore_processor_period": "900", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_run_sync_thread": "true", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_s3_auth_disable_signature_url": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_s3_auth_order": "sts, external, local", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_s3_auth_use_keystone": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_s3_auth_use_ldap": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_s3_auth_use_rados": "true", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_s3_auth_use_sts": "false", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_s3_client_max_sig_ver": "-1", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_s3_success_create_obj_status": "0", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_safe_max_objects_per_shard": "102400", 2026-02-25T08:41:33.650 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_scheduler_type": "throttler", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_script_uri": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_service_provider_name": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_shard_warning_threshold": "90.000000", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sts_client_id": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sts_client_secret": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sts_entry": "sts", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sts_key": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sts_max_session_duration": "43200", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sts_min_session_duration": "900", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sts_token_introspection_url": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_account_in_url": "false", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_auth_entry": "auth", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_auth_url": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_custom_header": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_enforce_content_length": "false", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_need_stats": "true", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_tenant_name": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_token_expiration": "86400", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_url": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_url_prefix": "swift", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_swift_versioning_enabled": "false", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_lease_period": "120", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_log_trim_interval": "1200", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_log_trim_max_buckets": "16", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_obj_etag_verify": "false", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_trace_history_size": "4096", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_trace_per_node_log_size": "32", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_thread_pool_size": "512", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_topic_persistency_max_retries": "0", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_topic_persistency_sleep_duration": "0", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_topic_persistency_time_to_live": "0", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_topic_require_publish_policy": "false", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_torrent_comment": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_torrent_createby": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_torrent_encoding": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_torrent_flag": "false", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_torrent_max_size": "5368709120", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_torrent_origin": "", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_torrent_sha_unit": "524288", 2026-02-25T08:41:33.651 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_torrent_tracker": "", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_trust_forwarded_https": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_usage_log_flush_threshold": "1024", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_usage_log_key_transition": "true", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_usage_log_tick_interval": "30", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_usage_max_shards": "32", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_usage_max_user_shards": "1", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_use_opa_authz": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_counters_cache": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_counters_cache_size": "10000", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_default_quota_max_objects": "-1", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_default_quota_max_size": "-1", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_max_buckets": "1000", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_policies_max_num": "100", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_quota_sync_idle_users": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_quota_sync_interval": "86400", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_quota_sync_wait_time": "86400", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_user_unique_email": "true", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_verify_ssl": "true", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_website_routing_rules_max_num": "50", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_zone": "", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_zone_id": "", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_zone_root_pool": ".rgw.root", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_zonegroup": "", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_zonegroup_id": "", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgwlc_auto_session_clear": "true", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rgwlc_skip_bucket_step": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_block_size": "4096", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_bloom_bits_per_key": "20", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cache_row_ratio": "0.000000", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cache_shard_bits": "4", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cache_size": "536870912", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cache_type": "binned_lru", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cf_compact_on_deletion": "true", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_collect_compaction_stats": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_collect_extended_stats": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_collect_memory_stats": "false", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_delete_range_threshold": "1048576", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_index_type": "binary_search", 2026-02-25T08:41:33.652 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_log_to_ceph_log": "true", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_metadata_block_size": "4096", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_partition_filters": "false", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_perf": "false", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "rotating_keys_bootstrap_timeout": "30", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "rotating_keys_renewal_timeout": "10", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "run_dir": "/var/run/ceph", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_block_create": "true", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_cachepin_2q_in_ratio": "0.500000", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_cachepin_2q_out_ratio": "0.500000", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_cachepin_size_pershard": "2147483648", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_cachepin_type": "LRU", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_cbjournal_size": "5368709120", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_cold_tier_generations": "3", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_data_delta_based_overwrite": "0", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_default_max_object_size": "16777216", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_default_object_metadata_reservation": "16777216", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_device_size": "53687091200", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_disable_end_to_end_data_protection": "true", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_full_integrity_check": "false", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_hot_tier_generations": "5", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_journal_batch_capacity": "16", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_journal_batch_flush_size": "16777216", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_journal_iodepth_limit": "5", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_main_device_type": "SSD", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_max_concurrent_transactions": "128", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_max_data_allocation_size": "0", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_segment_cleaner_gc_formula": "cost_benefit", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "seastore_segment_size": "67108864", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "service_unique_id": "", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "set_keepcaps": "false", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "setgroup": "", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "setuser": "", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "setuser_match_path": "", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "subv_metrics_window_interval": "30", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "target_max_misplaced_ratio": "0.050000", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "thp": "false", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "threadpool_default_timeout": "60", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "threadpool_empty_queue_max_wait": "2", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "throttler_perf_counter": "true", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "tmp_dir": "/tmp", 2026-02-25T08:41:33.653 INFO:teuthology.orchestra.run.trial042.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2026-02-25T08:41:33.654 INFO:teuthology.orchestra.run.trial042.stdout: "uadk_compressor_enabled": "false", 2026-02-25T08:41:33.654 INFO:teuthology.orchestra.run.trial042.stdout: "uadk_wd_sync_ctx_num": "2" 2026-02-25T08:41:33.654 INFO:teuthology.orchestra.run.trial042.stdout:} 2026-02-25T08:41:33.655 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2026-02-25T08:41:33.729 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.730+0000 7f36c1c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.732 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.734+0000 7f36c1c1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.741 INFO:teuthology.orchestra.run.trial042.stdout:{} 2026-02-25T08:41:33.741 INFO:teuthology.orchestra.run.trial042.stderr:mon_osd_down_out_interval = '' (not observed, change may require restart) 2026-02-25T08:41:33.747 DEBUG:teuthology.orchestra.run.trial042:> type ceph-objectstore-tool 2026-02-25T08:41:33.762 INFO:teuthology.orchestra.run.trial042.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2026-02-25T08:41:33.763 DEBUG:teuthology.orchestra.run.trial066:> type ceph-objectstore-tool 2026-02-25T08:41:33.779 INFO:teuthology.orchestra.run.trial066.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2026-02-25T08:41:33.779 INFO:teuthology.run_tasks:Running task radosbench... 2026-02-25T08:41:33.792 INFO:tasks.radosbench:Beginning radosbench... 2026-02-25T08:41:33.792 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2026-02-25T08:41:33.792 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2026-02-25T08:41:33.792 INFO:tasks.thrashosds.thrasher:starting do_thrash 2026-02-25T08:41:33.792 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3] 2026-02-25T08:41:33.792 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2026-02-25T08:41:33.792 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3] 2026-02-25T08:41:33.792 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-25T08:41:33.792 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:41:33.793 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2026-02-25T08:41:33.793 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:41:33.861 INFO:tasks.ceph.osd.1:Stopped 2026-02-25T08:41:33.879 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.880+0000 7f8d2a9dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.882 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.884+0000 7f8d2a9dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.882 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.880+0000 7fcb3accb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:33.885 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:33.888+0000 7fcb3accb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:34.263 INFO:teuthology.orchestra.run.trial042.stderr:pool 'unique_pool_0' created 2026-02-25T08:41:34.276 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2026-02-25T08:41:34.394 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:34.391+0000 7f95fb10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:34.396 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:34.399+0000 7f95fb10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:35.266 INFO:teuthology.orchestra.run.trial042.stderr:enabled application 'rados' on pool 'unique_pool_0' 2026-02-25T08:41:36.281 DEBUG:teuthology.orchestra.run.trial042:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -t 16 -b 65536 --object-size 65536 -p unique_pool_0 bench 150 write' 2026-02-25T08:41:36.284 INFO:teuthology.run_tasks:Running task admin_socket... 2026-02-25T08:41:36.304 DEBUG:tasks.admin_socket:Running admin socket tests on client.0 2026-02-25T08:41:36.304 DEBUG:teuthology.orchestra.run.trial042:> mkdir -- /home/ubuntu/cephtest/admin_socket_client.0 && while test '!' -e /var/run/ceph/ceph-client.0.asok ; do sleep 1 ; done 2026-02-25T08:41:36.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout:hints = 1 2026-02-25T08:41:36.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 150 seconds or 0 objects 2026-02-25T08:41:36.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Object prefix: benchmark_data_trial042_42290 2026-02-25T08:41:36.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-25T08:41:36.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 0 0 0 0 0 0 - 0 2026-02-25T08:41:37.322 DEBUG:tasks.admin_socket:Testing objecter_requests with config {'test': 'http://git.ceph.com/?p={repo};a=blob_plain;f=src/test/admin_socket/objecter_requests;hb={branch}', 'branch': 'wip-logstoremanager'} 2026-02-25T08:41:37.322 DEBUG:teuthology.orchestra.run.trial042:> wget -q -O /home/ubuntu/cephtest/admin_socket_client.0/objecter_requests -- 'http://git.ceph.com/?p=ceph-ci.git;a=blob_plain;f=src/test/admin_socket/objecter_requests;hb=wip-logstoremanager' && chmod u=rx -- /home/ubuntu/cephtest/admin_socket_client.0/objecter_requests 2026-02-25T08:41:37.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 1 16 20 4 0.249989 0.25 0.0117065 0.0236081 2026-02-25T08:41:37.525 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --admin-daemon /var/run/ceph/ceph-client.0.asok objecter_requests 2026-02-25T08:41:37.597 INFO:teuthology.orchestra.run.trial042.stdout:{ 2026-02-25T08:41:37.597 INFO:teuthology.orchestra.run.trial042.stdout: "ops": [ 2026-02-25T08:41:37.597 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.597 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 6, 2026-02-25T08:41:37.597 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.2733d2a1", 2026-02-25T08:41:37.597 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 0, 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object5", 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object5", 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.598 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334936+0000", 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 10, 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.d4301171", 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 0, 2026-02-25T08:41:37.599 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object9", 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object9", 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.600 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334966+0000", 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 12, 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.ee690e58", 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 1, 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object11", 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.601 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object11", 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334973+0000", 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.602 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 17, 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.34aa4e22", 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 1, 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object16", 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object16", 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.603 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2170.012281s", 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2419732880000001, 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.357698+0000", 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.604 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 1, 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.4f4e554f", 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 2, 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object0", 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object0", 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.605 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334877+0000", 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 2, 2026-02-25T08:41:37.606 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.605c6d16", 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 2, 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object1", 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object1", 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.607 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334918+0000", 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 3, 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.513d5d1e", 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 2, 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object2", 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.608 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object2", 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334924+0000", 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.609 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 4, 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.6b45f2ce", 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 2, 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object3", 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object3", 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.610 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334929+0000", 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.611 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 8, 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.1637454f", 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 2, 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object7", 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object7", 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.612 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334958+0000", 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 11, 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.28f52906", 2026-02-25T08:41:37.613 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 2, 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object10", 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object10", 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2649727930000001, 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.614 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334969+0000", 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 13, 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.f8b589e", 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 2, 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object12", 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.615 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object12", 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.265972772, 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334982+0000", 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.616 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 7, 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.9bf08799", 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 3, 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object6", 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object6", 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.617 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.265972772, 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334947+0000", 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.618 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 9, 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.772ed37", 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 3, 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object8", 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object8", 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.619 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.265972772, 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334963+0000", 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 15, 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.36b0752c", 2026-02-25T08:41:37.620 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 3, 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object14", 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object14", 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2169.989281s", 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.265972772, 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.621 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.334986+0000", 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 19, 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.c939bdc0", 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 3, 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object18", 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object18", 2026-02-25T08:41:37.622 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2170.021280s", 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2339734600000001, 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.366639+0000", 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.623 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: }, 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: { 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "tid": 20, 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "pg": "3.20f24809", 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "osd": 3, 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "object_id": "benchmark_data_trial042_42290_object19", 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "object_locator": "@3", 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_id": "benchmark_data_trial042_42290_object19", 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "target_object_locator": "@3", 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "paused": 0, 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "used_replica": 0, 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "precalc_pgid": 0, 2026-02-25T08:41:37.624 INFO:teuthology.orchestra.run.trial042.stdout: "last_sent": "2170.029280s", 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "age": 1.2259736320000001, 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "attempts": 1, 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "snapid": "head", 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "snap_context": "0=[]", 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "mtime": "2026-02-25T08:41:36.375019+0000", 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "osd_ops": [ 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "set-alloc-hint object_size 65536 write_size 65536", 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "write 0~65536 in=65536b" 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: ] 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: } 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: ], 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "linger_ops": [], 2026-02-25T08:41:37.625 INFO:teuthology.orchestra.run.trial042.stdout: "pool_ops": [], 2026-02-25T08:41:37.626 INFO:teuthology.orchestra.run.trial042.stdout: "pool_stat_ops": [], 2026-02-25T08:41:37.626 INFO:teuthology.orchestra.run.trial042.stdout: "statfs_ops": [], 2026-02-25T08:41:37.626 INFO:teuthology.orchestra.run.trial042.stdout: "command_ops": [] 2026-02-25T08:41:37.626 INFO:teuthology.orchestra.run.trial042.stdout:} 2026-02-25T08:41:37.627 DEBUG:tasks.admin_socket:admin socket command objecter_requests returned { "ops": [ { "tid": 6, "pg": "3.2733d2a1", "osd": 0, "object_id": "benchmark_data_trial042_42290_object5", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object5", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334936+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 10, "pg": "3.d4301171", "osd": 0, "object_id": "benchmark_data_trial042_42290_object9", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object9", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334966+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 12, "pg": "3.ee690e58", "osd": 1, "object_id": "benchmark_data_trial042_42290_object11", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object11", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334973+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 17, "pg": "3.34aa4e22", "osd": 1, "object_id": "benchmark_data_trial042_42290_object16", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object16", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2170.012281s", "age": 1.2419732880000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.357698+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 1, "pg": "3.4f4e554f", "osd": 2, "object_id": "benchmark_data_trial042_42290_object0", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object0", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334877+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 2, "pg": "3.605c6d16", "osd": 2, "object_id": "benchmark_data_trial042_42290_object1", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object1", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334918+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 3, "pg": "3.513d5d1e", "osd": 2, "object_id": "benchmark_data_trial042_42290_object2", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object2", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334924+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 4, "pg": "3.6b45f2ce", "osd": 2, "object_id": "benchmark_data_trial042_42290_object3", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object3", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334929+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 8, "pg": "3.1637454f", "osd": 2, "object_id": "benchmark_data_trial042_42290_object7", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object7", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334958+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 11, "pg": "3.28f52906", "osd": 2, "object_id": "benchmark_data_trial042_42290_object10", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object10", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.2649727930000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334969+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 13, "pg": "3.f8b589e", "osd": 2, "object_id": "benchmark_data_trial042_42290_object12", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object12", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.265972772, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334982+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 7, "pg": "3.9bf08799", "osd": 3, "object_id": "benchmark_data_trial042_42290_object6", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object6", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.265972772, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334947+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 9, "pg": "3.772ed37", "osd": 3, "object_id": "benchmark_data_trial042_42290_object8", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object8", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.265972772, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334963+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 15, "pg": "3.36b0752c", "osd": 3, "object_id": "benchmark_data_trial042_42290_object14", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object14", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2169.989281s", "age": 1.265972772, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.334986+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 19, "pg": "3.c939bdc0", "osd": 3, "object_id": "benchmark_data_trial042_42290_object18", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object18", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2170.021280s", "age": 1.2339734600000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.366639+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] }, { "tid": 20, "pg": "3.20f24809", "osd": 3, "object_id": "benchmark_data_trial042_42290_object19", "object_locator": "@3", "target_object_id": "benchmark_data_trial042_42290_object19", "target_object_locator": "@3", "paused": 0, "used_replica": 0, "precalc_pgid": 0, "last_sent": "2170.029280s", "age": 1.2259736320000001, "attempts": 1, "snapid": "head", "snap_context": "0=[]", "mtime": "2026-02-25T08:41:36.375019+0000", "osd_ops": [ "set-alloc-hint object_size 65536 write_size 65536", "write 0~65536 in=65536b" ] } ], "linger_ops": [], "pool_ops": [], "pool_stat_ops": [], "statfs_ops": [], "command_ops": [] } 2026-02-25T08:41:37.628 DEBUG:teuthology.orchestra.run.trial042:> /home/ubuntu/cephtest/admin_socket_client.0/objecter_requests 2026-02-25T08:41:37.658 DEBUG:teuthology.orchestra.run.trial042:> rm -rf -- /home/ubuntu/cephtest/admin_socket_client.0 2026-02-25T08:41:37.716 DEBUG:teuthology.parallel:result is None 2026-02-25T08:41:37.716 DEBUG:teuthology.run_tasks:Unwinding manager radosbench 2026-02-25T08:41:37.732 INFO:tasks.radosbench:joining radosbench (timing out after 4800s) 2026-02-25T08:41:37.733 INFO:teuthology.orchestra.run:waiting for 4800 2026-02-25T08:41:38.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 2 16 20 4 0.124993 0 - 0.0236081 2026-02-25T08:41:38.861 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2026-02-25T08:41:38.861 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2026-02-25T08:41:38.862 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:41:38.939 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:38.939+0000 7fc851e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:38.941 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:41:38.943+0000 7fc851e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:39.052 INFO:teuthology.orchestra.run.trial042.stdout: 2026-02-25T08:41:39.053 INFO:teuthology.orchestra.run.trial042.stderr:dumped all 2026-02-25T08:41:39.061 INFO:teuthology.orchestra.run.trial042.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2026-02-25T08:41:38.418116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":852531,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":85,"num_write_kb":1364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":209715136,"kb_used":83891816,"kb_used_data":83891816,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":125823320,"statfs":{"total":214748299264,"available":128843079680,"internally_reserved":0,"allocated":85905219584,"data_stored":85905219584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001148"},"pg_stats":[{"pgid":"3.f","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.278842+0000","last_change":"2026-02-25T08:41:34.336591+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.278842+0000","last_undegraded":"2026-02-25T08:41:35.278842+0000","last_fullsized":"2026-02-25T08:41:35.278842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.277475+0000","last_change":"2026-02-25T08:41:34.326518+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.277475+0000","last_undegraded":"2026-02-25T08:41:35.277475+0000","last_fullsized":"2026-02-25T08:41:35.277475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.280117+0000","last_change":"2026-02-25T08:41:35.318808+0000","last_active":"2026-02-25T08:41:36.280117+0000","last_peered":"2026-02-25T08:41:36.280117+0000","last_clean":"2026-02-25T08:41:36.280117+0000","last_became_active":"2026-02-25T08:41:35.314758+0000","last_became_peered":"2026-02-25T08:41:35.314758+0000","last_unstale":"2026-02-25T08:41:36.280117+0000","last_undegraded":"2026-02-25T08:41:36.280117+0000","last_fullsized":"2026-02-25T08:41:36.280117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.277348+0000","last_change":"2026-02-25T08:41:34.325900+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.277348+0000","last_undegraded":"2026-02-25T08:41:35.277348+0000","last_fullsized":"2026-02-25T08:41:35.277348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-25T08:41:34.262572+0000","last_change":"2026-02-25T08:41:34.262572+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:34.262572+0000","last_undegraded":"2026-02-25T08:41:34.262572+0000","last_fullsized":"2026-02-25T08:41:34.262572+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.278530+0000","last_change":"2026-02-25T08:41:34.315073+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.278530+0000","last_undegraded":"2026-02-25T08:41:35.278530+0000","last_fullsized":"2026-02-25T08:41:35.278530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.277086+0000","last_change":"2026-02-25T08:41:34.304678+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.277086+0000","last_undegraded":"2026-02-25T08:41:35.277086+0000","last_fullsized":"2026-02-25T08:41:35.277086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-25T08:41:34.262572+0000","last_change":"2026-02-25T08:41:34.262572+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:34.262572+0000","last_undegraded":"2026-02-25T08:41:34.262572+0000","last_fullsized":"2026-02-25T08:41:34.262572+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":100,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.278424+0000","last_change":"2026-02-25T08:41:19.740059+0000","last_active":"2026-02-25T08:41:36.278424+0000","last_peered":"2026-02-25T08:41:36.278424+0000","last_clean":"2026-02-25T08:41:36.278424+0000","last_became_active":"2026-02-25T08:41:19.730156+0000","last_became_peered":"2026-02-25T08:41:19.730156+0000","last_unstale":"2026-02-25T08:41:36.278424+0000","last_undegraded":"2026-02-25T08:41:36.278424+0000","last_fullsized":"2026-02-25T08:41:36.278424+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:18.697053+0000","last_clean_scrub_stamp":"2026-02-25T08:41:18.697053+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":26,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.280689+0000","last_change":"2026-02-25T08:41:28.261332+0000","last_active":"2026-02-25T08:41:36.280689+0000","last_peered":"2026-02-25T08:41:36.280689+0000","last_clean":"2026-02-25T08:41:36.280689+0000","last_became_active":"2026-02-25T08:41:26.283745+0000","last_became_peered":"2026-02-25T08:41:26.283745+0000","last_unstale":"2026-02-25T08:41:36.280689+0000","last_undegraded":"2026-02-25T08:41:36.280689+0000","last_fullsized":"2026-02-25T08:41:36.280689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2026-02-25T08:41:34.262572+0000","last_change":"2026-02-25T08:41:34.262572+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:34.262572+0000","last_undegraded":"2026-02-25T08:41:34.262572+0000","last_fullsized":"2026-02-25T08:41:34.262572+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.280362+0000","last_change":"2026-02-25T08:41:28.261595+0000","last_active":"2026-02-25T08:41:36.280362+0000","last_peered":"2026-02-25T08:41:36.280362+0000","last_clean":"2026-02-25T08:41:36.280362+0000","last_became_active":"2026-02-25T08:41:26.301018+0000","last_became_peered":"2026-02-25T08:41:26.301018+0000","last_unstale":"2026-02-25T08:41:36.280362+0000","last_undegraded":"2026-02-25T08:41:36.280362+0000","last_fullsized":"2026-02-25T08:41:36.280362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.281615+0000","last_change":"2026-02-25T08:41:34.284744+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.281615+0000","last_undegraded":"2026-02-25T08:41:35.281615+0000","last_fullsized":"2026-02-25T08:41:35.281615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.260909+0000","last_change":"2026-02-25T08:41:28.263380+0000","last_active":"2026-02-25T08:41:28.260909+0000","last_peered":"2026-02-25T08:41:28.260909+0000","last_clean":"2026-02-25T08:41:28.260909+0000","last_became_active":"2026-02-25T08:41:26.287664+0000","last_became_peered":"2026-02-25T08:41:26.287664+0000","last_unstale":"2026-02-25T08:41:28.260909+0000","last_undegraded":"2026-02-25T08:41:28.260909+0000","last_fullsized":"2026-02-25T08:41:28.260909+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0],"acting":[1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.275722+0000","last_change":"2026-02-25T08:41:34.279135+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.275722+0000","last_undegraded":"2026-02-25T08:41:35.275722+0000","last_fullsized":"2026-02-25T08:41:35.275722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":27,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.280129+0000","last_change":"2026-02-25T08:41:28.261547+0000","last_active":"2026-02-25T08:41:36.280129+0000","last_peered":"2026-02-25T08:41:36.280129+0000","last_clean":"2026-02-25T08:41:36.280129+0000","last_became_active":"2026-02-25T08:41:26.280456+0000","last_became_peered":"2026-02-25T08:41:26.280456+0000","last_unstale":"2026-02-25T08:41:36.280129+0000","last_undegraded":"2026-02-25T08:41:36.280129+0000","last_fullsized":"2026-02-25T08:41:36.280129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.279704+0000","last_change":"2026-02-25T08:41:35.323393+0000","last_active":"2026-02-25T08:41:36.279704+0000","last_peered":"2026-02-25T08:41:36.279704+0000","last_clean":"2026-02-25T08:41:36.279704+0000","last_became_active":"2026-02-25T08:41:35.313337+0000","last_became_peered":"2026-02-25T08:41:35.313337+0000","last_unstale":"2026-02-25T08:41:36.279704+0000","last_undegraded":"2026-02-25T08:41:36.279704+0000","last_fullsized":"2026-02-25T08:41:36.279704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.280397+0000","last_change":"2026-02-25T08:41:35.321428+0000","last_active":"2026-02-25T08:41:36.280397+0000","last_peered":"2026-02-25T08:41:36.280397+0000","last_clean":"2026-02-25T08:41:36.280397+0000","last_became_active":"2026-02-25T08:41:35.315349+0000","last_became_peered":"2026-02-25T08:41:35.315349+0000","last_unstale":"2026-02-25T08:41:36.280397+0000","last_undegraded":"2026-02-25T08:41:36.280397+0000","last_fullsized":"2026-02-25T08:41:36.280397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T08:41:28.263550+0000","last_change":"2026-02-25T08:41:28.265491+0000","last_active":"2026-02-25T08:41:28.263550+0000","last_peered":"2026-02-25T08:41:28.263550+0000","last_clean":"2026-02-25T08:41:28.263550+0000","last_became_active":"2026-02-25T08:41:26.285926+0000","last_became_peered":"2026-02-25T08:41:26.285926+0000","last_unstale":"2026-02-25T08:41:28.263550+0000","last_undegraded":"2026-02-25T08:41:28.263550+0000","last_fullsized":"2026-02-25T08:41:28.263550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"21'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.337888+0000","last_change":"2026-02-25T08:41:35.308632+0000","last_active":"2026-02-25T08:41:36.337888+0000","last_peered":"2026-02-25T08:41:36.337888+0000","last_clean":"2026-02-25T08:41:36.337888+0000","last_became_active":"2026-02-25T08:41:35.301747+0000","last_became_peered":"2026-02-25T08:41:35.301747+0000","last_unstale":"2026-02-25T08:41:36.337888+0000","last_undegraded":"2026-02-25T08:41:36.337888+0000","last_fullsized":"2026-02-25T08:41:36.337888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.280613+0000","last_change":"2026-02-25T08:41:28.261618+0000","last_active":"2026-02-25T08:41:36.280613+0000","last_peered":"2026-02-25T08:41:36.280613+0000","last_clean":"2026-02-25T08:41:36.280613+0000","last_became_active":"2026-02-25T08:41:26.284252+0000","last_became_peered":"2026-02-25T08:41:26.284252+0000","last_unstale":"2026-02-25T08:41:36.280613+0000","last_undegraded":"2026-02-25T08:41:36.280613+0000","last_fullsized":"2026-02-25T08:41:36.280613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2],"acting":[3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.278712+0000","last_change":"2026-02-25T08:41:34.293349+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.278712+0000","last_undegraded":"2026-02-25T08:41:35.278712+0000","last_fullsized":"2026-02-25T08:41:35.278712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.279183+0000","last_change":"2026-02-25T08:41:28.261605+0000","last_active":"2026-02-25T08:41:36.279183+0000","last_peered":"2026-02-25T08:41:36.279183+0000","last_clean":"2026-02-25T08:41:36.279183+0000","last_became_active":"2026-02-25T08:41:26.282840+0000","last_became_peered":"2026-02-25T08:41:26.282840+0000","last_unstale":"2026-02-25T08:41:36.279183+0000","last_undegraded":"2026-02-25T08:41:36.279183+0000","last_fullsized":"2026-02-25T08:41:36.279183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":3,"reported_epoch":21,"state":"creating+peering","last_fresh":"2026-02-25T08:41:35.277461+0000","last_change":"2026-02-25T08:41:34.316817+0000","last_active":"2026-02-25T08:41:34.262572+0000","last_peered":"2026-02-25T08:41:34.262572+0000","last_clean":"2026-02-25T08:41:34.262572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2026-02-25T08:41:35.277461+0000","last_undegraded":"2026-02-25T08:41:35.277461+0000","last_fullsized":"2026-02-25T08:41:35.277461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:34.262572+0000","last_clean_scrub_stamp":"2026-02-25T08:41:34.262572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":21,"state":"active+clean","last_fresh":"2026-02-25T08:41:36.279576+0000","last_change":"2026-02-25T08:41:28.261259+0000","last_active":"2026-02-25T08:41:36.279576+0000","last_peered":"2026-02-25T08:41:36.279576+0000","last_clean":"2026-02-25T08:41:36.279576+0000","last_became_active":"2026-02-25T08:41:26.297414+0000","last_became_peered":"2026-02-25T08:41:26.297414+0000","last_unstale":"2026-02-25T08:41:36.279576+0000","last_undegraded":"2026-02-25T08:41:36.279576+0000","last_fullsized":"2026-02-25T08:41:36.279576+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:25.238143+0000","last_clean_scrub_stamp":"2026-02-25T08:41:25.238143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":262144,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96631304192,"internally_reserved":0,"allocated":64429920256,"data_stored":64429920256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":39,"acting":39,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96632623104,"internally_reserved":0,"allocated":64428601344,"data_stored":64428601344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32210358272,"internally_reserved":0,"allocated":21476716544,"data_stored":21476716544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":3,"acting":3,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971972,"kb_used_data":20971972,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456812,"statfs":{"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973356,"kb_used_data":20973356,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31455428,"statfs":{"total":53687074816,"available":32210358272,"internally_reserved":0,"allocated":21476716544,"data_stored":21476716544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973228,"kb_used_data":20973228,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31455556,"statfs":{"total":53687074816,"available":32210489344,"internally_reserved":0,"allocated":21476585472,"data_stored":21476585472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640275,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973260,"kb_used_data":20973260,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31455524,"statfs":{"total":53687074816,"available":32210456576,"internally_reserved":0,"allocated":21476618240,"data_stored":21476618240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":3,"total":53687074816,"available":32210358272,"internally_reserved":0,"allocated":21476716544,"data_stored":21476716544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211775488,"internally_reserved":0,"allocated":21475299328,"data_stored":21475299328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32210489344,"internally_reserved":0,"allocated":21476585472,"data_stored":21476585472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":53687074816,"available":32210358272,"internally_reserved":0,"allocated":21476716544,"data_stored":21476716544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":53687074816,"available":32210456576,"internally_reserved":0,"allocated":21476618240,"data_stored":21476618240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":53687074816,"available":32210489344,"internally_reserved":0,"allocated":21476585472,"data_stored":21476585472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":53687074816,"available":32210358272,"internally_reserved":0,"allocated":21476716544,"data_stored":21476716544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:41:39.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 3 16 20 4 0.0833285 0 - 0.0236081 2026-02-25T08:41:40.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 4 16 20 4 0.0624963 0 - 0.0236081 2026-02-25T08:41:41.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 5 16 20 4 0.0499971 0 - 0.0236081 2026-02-25T08:41:42.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 6 16 20 4 0.0416642 0 - 0.0236081 2026-02-25T08:41:43.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 7 16 20 4 0.0357122 0 - 0.0236081 2026-02-25T08:41:44.064 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3] 2026-02-25T08:41:44.064 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2026-02-25T08:41:44.064 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2026-02-25T08:41:44.064 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-25T08:41:44.065 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-25T08:41:44.067 INFO:tasks.ceph.osd.1:Started 2026-02-25T08:41:44.068 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:44.139 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:44.142 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:44.142 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:44.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 8 16 20 4 0.0312482 0 - 0.0236081 2026-02-25T08:41:45.331 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 9 16 20 4 0.0277762 0 - 0.0236081 2026-02-25T08:41:46.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 10 16 20 4 0.0249985 0 - 0.0236081 2026-02-25T08:41:47.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 11 16 20 4 0.0227259 0 - 0.0236081 2026-02-25T08:41:48.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 12 16 20 4 0.0208321 0 - 0.0236081 2026-02-25T08:41:49.143 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:49.214 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:49.217 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:49.217 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:49.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 13 16 20 4 0.0192296 0 - 0.0236081 2026-02-25T08:41:50.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 14 16 20 4 0.0178561 0 - 0.0236081 2026-02-25T08:41:50.995 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2026-02-25T08:41:51.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 15 16 20 4 0.0166657 0 - 0.0236081 2026-02-25T08:41:52.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 16 16 20 4 0.0156241 0 - 0.0236081 2026-02-25T08:41:53.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 17 16 20 4 0.014705 0 - 0.0236081 2026-02-25T08:41:54.218 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:54.290 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:54.293 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:54.293 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:41:54.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 18 16 20 4 0.0138881 0 - 0.0236081 2026-02-25T08:41:55.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 19 16 20 4 0.0131571 0 - 0.0236081 2026-02-25T08:41:56.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout:2026-02-25T08:41:56.336037+0000 min lat: 0.0117065 max lat: 0.0316485 avg lat: 0.0236081 2026-02-25T08:41:56.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-25T08:41:56.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 20 16 20 4 0.0124993 0 - 0.0236081 2026-02-25T08:41:56.500 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~6s 2026-02-25T08:41:57.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 21 16 20 4 0.0119041 0 - 0.0236081 2026-02-25T08:41:58.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 22 16 20 4 0.011363 0 - 0.0236081 2026-02-25T08:41:59.205 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:41:59.283 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:41:59.284 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:41:59.294 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:41:59.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 23 16 20 4 0.0108689 0 - 0.0236081 2026-02-25T08:41:59.393 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:41:59.395 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:41:59.396 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:00.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 24 16 26 10 0.0260401 0.0163043 24.001 14.3972 2026-02-25T08:42:01.286 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:01.332 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 25 16 303 287 0.717457 17.3125 0.0207266 1.38933 2026-02-25T08:42:01.365 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:01.367+0000 7f00f4774640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:01.369 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:01.372+0000 7f00f4774640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:01.384 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:01.477 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:01.477 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:02.004 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~11s 2026-02-25T08:42:02.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 26 16 530 514 1.2355 14.1875 0.0273494 0.807057 2026-02-25T08:42:03.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 27 16 862 846 1.95821 20.75 0.0660633 0.51008 2026-02-25T08:42:03.478 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:03.559 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:03.560+0000 7f861a035640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:03.561 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:03.563+0000 7f861a035640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:03.583 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:03.703 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:03.703 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:04.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 28 16 1158 1142 2.54895 18.5 0.0236092 0.391647 2026-02-25T08:42:04.397 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:04.472 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:04.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:04.475 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:05.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 29 16 1515 1499 3.23041 22.3125 0.0219643 0.309216 2026-02-25T08:42:05.705 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:05.782 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:05.783+0000 7f0883a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:05.785 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:05.787+0000 7f0883a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:05.810 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:05.933 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:05.933 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:06.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 30 16 1858 1842 3.83726 21.4375 0.0412248 0.260381 2026-02-25T08:42:07.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 31 16 2195 2179 4.39287 21.0625 0.0324626 0.227385 2026-02-25T08:42:07.508 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~17s 2026-02-25T08:42:07.934 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:08.016 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:08.016+0000 7fc27fde4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:08.018 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:08.021+0000 7fc27fde4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:08.042 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:08.148 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:08.149 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:08.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 32 16 2533 2517 4.91571 21.125 0.069205 0.203296 2026-02-25T08:42:09.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 33 16 2903 2887 5.46747 23.125 0.0265793 0.182796 2026-02-25T08:42:09.476 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:09.553 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:09.556 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:09.557 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:10.150 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:10.228 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:10.229+0000 7f2bac7a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:10.231 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:10.233+0000 7f2bac7a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:10.247 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:10.314 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:10.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:10.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 34 16 3239 3223 5.92427 21 0.0430183 0.168543 2026-02-25T08:42:11.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 35 16 3560 3544 6.32819 20.0625 0.0368023 0.157887 2026-02-25T08:42:12.316 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:12.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 36 16 3892 3876 6.72875 20.75 0.133162 0.148459 2026-02-25T08:42:12.397 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:12.398+0000 7fd53958c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:12.400 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:12.402+0000 7fd53958c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:12.435 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:12.555 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:12.556 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:13.013 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~22s 2026-02-25T08:42:13.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 37 16 4216 4200 7.09416 20.25 0.049771 0.140854 2026-02-25T08:42:14.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 38 16 4564 4548 7.4798 21.75 0.0362689 0.13359 2026-02-25T08:42:14.556 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:14.558 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:14.631 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:14.634 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:14.634 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:14.638 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:14.638+0000 7f6167564640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:14.640 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:14.643+0000 7f6167564640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:14.670 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:14.763 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:14.764 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:15.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 39 16 4896 4880 7.82003 20.75 0.0782075 0.127755 2026-02-25T08:42:16.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout:2026-02-25T08:42:16.337344+0000 min lat: 0.0107423 max lat: 24.1039 avg lat: 0.122811 2026-02-25T08:42:16.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-25T08:42:16.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 40 16 5224 5208 8.137 20.5 0.0464976 0.122811 2026-02-25T08:42:16.765 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:16.846 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:16.847+0000 7f1f178cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:16.849 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:16.851+0000 7f1f178cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:16.868 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:16.961 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:16.961 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:17.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 41 16 5545 5529 8.42784 20.0625 0.0308294 0.118574 2026-02-25T08:42:18.333 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 42 16 5857 5841 8.69144 19.5 0.0363739 0.114982 2026-02-25T08:42:18.516 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~28s 2026-02-25T08:42:18.962 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:19.040 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:19.041+0000 7fa774fa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:19.042 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:19.045+0000 7fa774fa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:19.059 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:19.154 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:19.155 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:19.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 43 16 6176 6160 8.95295 19.9375 0.0255444 0.111549 2026-02-25T08:42:19.636 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:19.709 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:19.713 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:19.714 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:20.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 44 16 6490 6474 9.19546 19.625 0.0399068 0.108685 2026-02-25T08:42:21.155 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:21.232 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:21.233+0000 7f226da1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:21.235 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:21.237+0000 7f226da1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:21.251 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:21.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 45 16 6792 6776 9.41054 18.875 0.0646382 0.106169 2026-02-25T08:42:21.367 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:21.367 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:22.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 46 16 7123 7107 9.65566 20.6875 0.0294121 0.103506 2026-02-25T08:42:23.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 47 16 7445 7429 9.87839 20.125 0.0615059 0.101142 2026-02-25T08:42:23.368 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:23.447 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:23.448+0000 7fdc44ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:23.450 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:23.452+0000 7fdc44ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:23.469 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:23.566 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:23.566 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:24.020 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~33s 2026-02-25T08:42:24.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 48 16 7774 7758 10.1009 20.5625 0.0499549 0.0989385 2026-02-25T08:42:24.715 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:24.789 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:24.791 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:24.792 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:25.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 49 16 8108 8092 10.3208 20.875 0.051264 0.0968463 2026-02-25T08:42:25.567 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:25.646 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:25.647+0000 7f2dc625a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:25.649 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:25.651+0000 7f2dc625a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:25.668 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:25.766 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:25.766 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:26.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 50 16 8451 8435 10.5431 21.4375 0.0180046 0.0948017 2026-02-25T08:42:27.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 51 16 8779 8763 10.7383 20.5 0.0672752 0.0930679 2026-02-25T08:42:27.768 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:27.850 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:27.851+0000 7fedf07b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:27.852 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:27.854+0000 7fedf07b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:27.897 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:28.031 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:28.032 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:28.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 52 16 9124 9108 10.9465 21.5625 0.0251523 0.091282 2026-02-25T08:42:29.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 53 16 9445 9429 11.1184 20.0625 0.0776123 0.089872 2026-02-25T08:42:29.525 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~39s 2026-02-25T08:42:29.793 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:29.868 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:29.871 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:29.872 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:30.033 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:30.113 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:30.114+0000 7fa78919c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:30.116 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:30.118+0000 7fa78919c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:30.133 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:30.212 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:30.213 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:30.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 54 16 9774 9758 11.2933 20.5625 0.0293568 0.0884893 2026-02-25T08:42:31.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 55 16 10102 10086 11.4607 20.5 0.0752309 0.0871891 2026-02-25T08:42:32.214 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:32.295 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:32.296+0000 7fd2305fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:32.297 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:32.300+0000 7fd2305fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:32.313 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:32.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 56 16 10408 10392 11.5975 19.125 0.0563311 0.0861474 2026-02-25T08:42:32.444 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:32.444 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:33.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 57 16 10735 10719 11.7526 20.4375 0.0845148 0.0850312 2026-02-25T08:42:34.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 58 16 11073 11057 11.9142 21.125 0.037434 0.0838919 2026-02-25T08:42:34.446 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:34.524 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:34.525+0000 7f12e57dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:34.527 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:34.529+0000 7f12e57dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:34.544 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:34.612 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:34.613 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:34.872 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:34.947 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:34.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:34.951 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:35.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~44s 2026-02-25T08:42:35.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 59 16 11354 11338 12.0099 17.5625 0.0744734 0.0832308 2026-02-25T08:42:36.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout:2026-02-25T08:42:36.338404+0000 min lat: 0.0107423 max lat: 24.1039 avg lat: 0.0822937 2026-02-25T08:42:36.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-25T08:42:36.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 60 16 11672 11656 12.141 19.875 0.0440076 0.0822937 2026-02-25T08:42:36.614 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:36.692 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:36.695+0000 7ff730f6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:36.695 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:36.698+0000 7ff730f6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:36.713 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:36.842 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:36.843 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:37.334 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 61 16 11961 11945 12.238 18.0625 0.0573576 0.0816764 2026-02-25T08:42:38.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 62 16 12236 12220 12.3178 17.1875 0.0984818 0.0811398 2026-02-25T08:42:38.845 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:38.924 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:38.926+0000 7f355a39c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:38.926 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:38.929+0000 7f355a39c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:38.950 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:39.038 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:39.039 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:39.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 63 16 12522 12506 12.406 17.875 0.0511458 0.0805668 2026-02-25T08:42:39.952 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:40.025 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:40.028 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:40.029 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:40.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 64 16 12818 12802 12.5012 18.5 0.0206183 0.0799434 2026-02-25T08:42:40.534 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~50s 2026-02-25T08:42:41.040 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:41.122 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:41.124+0000 7f574665a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:41.125 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:41.129+0000 7f574665a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:41.153 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:41.230 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:41.231 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:41.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 65 16 13097 13081 12.5772 17.4375 0.0250333 0.0794761 2026-02-25T08:42:42.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 66 16 13394 13378 12.6678 18.5625 0.0177959 0.0788881 2026-02-25T08:42:43.232 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:43.312 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:43.314+0000 7f3e77ffd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:43.314 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:43.318+0000 7f3e77ffd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:43.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 67 16 13677 13661 12.7427 17.6875 0.0661629 0.0784367 2026-02-25T08:42:43.335 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:43.435 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:43.436 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:44.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 68 16 13958 13942 12.8136 17.5625 0.188335 0.0779808 2026-02-25T08:42:45.029 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:45.103 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:45.106 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:45.106 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:45.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 69 16 14248 14232 12.8906 18.125 0.0562072 0.0775511 2026-02-25T08:42:45.436 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:45.514 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:45.516+0000 7f6b5c251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:45.516 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:45.520+0000 7f6b5c251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:45.539 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:45.621 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:45.622 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:46.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~55s 2026-02-25T08:42:46.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 70 16 14527 14511 12.9555 17.4375 0.0543435 0.0771308 2026-02-25T08:42:47.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 71 16 14819 14803 13.03 18.25 0.0270246 0.0767071 2026-02-25T08:42:47.623 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:47.701 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:47.703+0000 7f5fa550b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:47.703 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:47.706+0000 7f5fa550b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:47.718 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:47.786 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:47.787 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:48.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 72 16 15087 15071 13.0817 16.75 0.0547434 0.0763971 2026-02-25T08:42:49.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 73 16 15376 15360 13.1499 18.0625 0.0214679 0.0760208 2026-02-25T08:42:49.788 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:49.869 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:49.870+0000 7f46ed05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:49.872 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:49.875+0000 7f46ed05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:49.891 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:49.984 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:49.985 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:50.107 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:50.180 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:50.183 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:50.184 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:50.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 74 16 15693 15677 13.2399 19.8125 0.0248526 0.0754979 2026-02-25T08:42:51.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 75 16 15979 15963 13.3017 17.875 0.0311577 0.0751445 2026-02-25T08:42:51.542 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~61s 2026-02-25T08:42:51.986 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:52.066 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:52.067+0000 7f7251162640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:52.069 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:52.071+0000 7f7251162640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:52.087 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:52.180 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:52.181 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:52.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 76 16 16262 16246 13.3594 17.6875 0.0448317 0.0747958 2026-02-25T08:42:53.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 77 16 16537 16521 13.4091 17.1875 0.0644564 0.0745364 2026-02-25T08:42:54.182 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:54.262 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:54.263+0000 7f7c4cc1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:54.265 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:54.267+0000 7f7c4cc1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:54.285 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:54.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 78 16 16828 16812 13.4704 18.1875 0.0444865 0.0741981 2026-02-25T08:42:54.390 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:54.391 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:55.186 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:42:55.258 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:42:55.261 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:42:55.261 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:42:55.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 79 16 17108 17092 13.5214 17.5 0.0443599 0.0739194 2026-02-25T08:42:56.335 INFO:tasks.radosbench.radosbench.0.trial042.stdout:2026-02-25T08:42:56.339581+0000 min lat: 0.0107423 max lat: 24.1039 avg lat: 0.0736824 2026-02-25T08:42:56.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-25T08:42:56.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 80 16 17382 17366 13.5664 17.125 0.0557995 0.0736824 2026-02-25T08:42:56.392 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:42:56.470 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:56.471+0000 7f6250cc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:56.473 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:56.475+0000 7f6250cc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:56.489 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:56.583 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:56.583 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:57.045 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~66s 2026-02-25T08:42:57.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 81 16 17646 17630 13.6026 16.5 0.13403 0.0734662 2026-02-25T08:42:58.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 82 16 17916 17900 13.6425 16.875 0.0185941 0.0732654 2026-02-25T08:42:58.585 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:42:58.662 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:58.663+0000 7f7e5a61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:58.665 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:42:58.668+0000 7f7e5a61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:42:58.680 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:42:58.812 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:42:58.813 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:42:59.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 83 16 18191 18175 13.6852 17.1875 0.0844348 0.0730458 2026-02-25T08:43:00.262 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:00.336 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:00.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 84 16 18463 18447 13.7246 17 0.100062 0.0728273 2026-02-25T08:43:00.338 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:00.339 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:00.813 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:00.892 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:00.893+0000 7f22b965a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:00.895 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:00.897+0000 7f22b965a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:00.914 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:01.019 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:01.020 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:01.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 85 16 18749 18733 13.7734 17.875 0.0840739 0.0725738 2026-02-25T08:43:02.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 86 16 19019 19003 13.8095 16.875 0.0545437 0.0723795 2026-02-25T08:43:02.548 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~72s 2026-02-25T08:43:03.021 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:03.102 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:03.102+0000 7fc5e1ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:03.104 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:03.106+0000 7fc5e1ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:03.120 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:03.216 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:03.216 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:03.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 87 16 19312 19296 13.8612 18.3125 0.134185 0.0721073 2026-02-25T08:43:04.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 88 16 19589 19573 13.9004 17.3125 0.0634963 0.0719142 2026-02-25T08:43:05.217 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:05.299 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:05.299+0000 7fa2c54cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:05.301 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:05.303+0000 7fa2c54cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:05.318 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:05.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 89 16 19880 19864 13.9486 18.1875 0.0304423 0.0716609 2026-02-25T08:43:05.340 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:05.408 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:05.408 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:05.414 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:05.416 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:05.417 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:06.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 90 16 20171 20155 13.9957 18.1875 0.0279899 0.0714258 2026-02-25T08:43:07.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 91 16 20456 20440 14.0376 17.8125 0.0682254 0.0712083 2026-02-25T08:43:07.409 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:07.490 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:07.491+0000 7feb270cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:07.493 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:07.495+0000 7feb270cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:07.510 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:07.663 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:07.664 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:08.052 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~77s 2026-02-25T08:43:08.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 92 16 20738 20722 14.0766 17.625 0.0481479 0.0710195 2026-02-25T08:43:09.336 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 93 16 21017 21001 14.1127 17.4375 0.0361399 0.0708388 2026-02-25T08:43:09.664 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:09.742 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:09.743+0000 7fd38105a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:09.744 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:09.747+0000 7fd38105a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:09.776 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:09.883 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:09.883 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:10.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 94 16 21270 21254 14.1308 15.8125 0.102561 0.0707228 2026-02-25T08:43:10.418 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:10.494 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:10.497 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:10.497 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:11.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 95 16 21546 21530 14.1636 17.25 0.0948286 0.0705797 2026-02-25T08:43:11.885 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:11.964 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:11.965+0000 7facf895b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:11.966 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:11.969+0000 7facf895b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:11.992 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:12.094 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:12.095 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:12.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 96 15 21844 21829 14.2107 18.6875 0.037073 0.0703371 2026-02-25T08:43:13.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 97 16 22125 22109 14.2446 17.5 0.027834 0.0701746 2026-02-25T08:43:13.555 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~83s 2026-02-25T08:43:14.096 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:14.179 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:14.180+0000 7f2abd8b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:14.182 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:14.184+0000 7f2abd8b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:14.222 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:14.283 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:14.283 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:14.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 98 16 22412 22396 14.2823 17.9375 0.030919 0.0699795 2026-02-25T08:43:15.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 99 16 22707 22691 14.3243 18.4375 0.0608032 0.0697925 2026-02-25T08:43:15.499 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:15.572 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:15.575 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:15.575 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:16.284 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:16.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout:2026-02-25T08:43:16.340898+0000 min lat: 0.0107423 max lat: 24.1039 avg lat: 0.0696438 2026-02-25T08:43:16.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-25T08:43:16.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 100 16 22984 22968 14.3541 17.3125 0.0465484 0.0696438 2026-02-25T08:43:16.365 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:16.365+0000 7f9560ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:16.368 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:16.370+0000 7f9560ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:16.388 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:16.479 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:16.479 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:17.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 101 16 23272 23256 14.3902 18 0.0495672 0.0694621 2026-02-25T08:43:18.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 102 16 23557 23541 14.4238 17.8125 0.0801322 0.0693034 2026-02-25T08:43:18.481 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:18.561 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:18.563+0000 7fda6edcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:18.564 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:18.566+0000 7fda6edcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:18.585 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:18.737 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:18.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:19.059 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~88s 2026-02-25T08:43:19.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 103 16 23821 23805 14.4439 16.5 0.0681001 0.0692104 2026-02-25T08:43:20.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 104 16 24093 24077 14.4685 17 0.0356064 0.0690892 2026-02-25T08:43:20.576 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:20.649 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:20.652 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:20.653 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:20.737 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:20.820 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:20.821+0000 7fae8da5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:20.823 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:20.825+0000 7fae8da5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:20.846 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:20.970 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:20.971 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:21.337 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 105 16 24360 24344 14.4896 16.6875 0.0276641 0.0689894 2026-02-25T08:43:22.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 106 16 24652 24636 14.5251 18.25 0.0676712 0.0688265 2026-02-25T08:43:22.972 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:23.052 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:23.054+0000 7f4f2548b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:23.055 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:23.058+0000 7f4f2548b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:23.081 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:23.181 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:23.182 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:23.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 107 16 24927 24911 14.5499 17.1875 0.0524058 0.0687043 2026-02-25T08:43:24.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 108 16 25211 25195 14.5796 17.75 0.0464952 0.068571 2026-02-25T08:43:24.563 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~94s 2026-02-25T08:43:25.183 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:25.264 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:25.266+0000 7fa41405a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:25.267 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:25.270+0000 7fa41405a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:25.285 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:25.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 109 16 25463 25447 14.5903 15.75 0.0632449 0.0685138 2026-02-25T08:43:25.357 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:25.357 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:25.654 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:25.727 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:25.730 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:25.731 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:26.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 110 16 25712 25696 14.5991 15.5625 0.0717638 0.0684639 2026-02-25T08:43:27.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 111 16 25979 25963 14.6179 16.6875 0.0428711 0.0683874 2026-02-25T08:43:27.358 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:27.437 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:27.439+0000 7fb7b090b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:27.440 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:27.443+0000 7fb7b090b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:27.482 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:27.612 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:27.613 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:28.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 112 16 26248 26232 14.6375 16.8125 0.132118 0.0682892 2026-02-25T08:43:29.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 113 16 26487 26471 14.6401 14.9375 0.0459014 0.0682742 2026-02-25T08:43:29.614 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:29.693 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:29.695+0000 7fe026e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:29.696 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:29.699+0000 7fe026e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:29.723 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:29.862 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:29.863 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:30.067 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~99s 2026-02-25T08:43:30.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 114 16 26774 26758 14.6691 17.9375 0.063416 0.0681528 2026-02-25T08:43:30.732 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:30.803 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:30.806 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:30.806 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:31.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 115 16 27023 27007 14.6768 15.5625 0.0702408 0.0680969 2026-02-25T08:43:31.864 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:31.944 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:31.946+0000 7f125230b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:31.947 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:31.950+0000 7f125230b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:31.966 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:32.046 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:32.047 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:32.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 116 16 27291 27275 14.6947 16.75 0.0475089 0.0680329 2026-02-25T08:43:33.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 117 16 27554 27538 14.7096 16.4375 0.0249543 0.0679598 2026-02-25T08:43:34.047 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:34.129 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:34.131+0000 7fb69f10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:34.132 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:34.136+0000 7fb69f10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:34.167 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:34.305 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:34.305 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:34.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 118 16 27844 27828 14.7385 18.125 0.0695595 0.0678312 2026-02-25T08:43:35.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 119 16 28119 28103 14.7591 17.1875 0.0714068 0.0677311 2026-02-25T08:43:35.570 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~105s 2026-02-25T08:43:35.807 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:35.880 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:35.883 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:35.883 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:36.307 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:36.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout:2026-02-25T08:43:36.342214+0000 min lat: 0.0107423 max lat: 24.1039 avg lat: 0.0676769 2026-02-25T08:43:36.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-25T08:43:36.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 120 16 28378 28362 14.771 16.1875 0.0473862 0.0676769 2026-02-25T08:43:36.387 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:36.388+0000 7f7abb96f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:36.389 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:36.392+0000 7f7abb96f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:36.429 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:36.502 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:36.503 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:37.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 121 16 28640 28624 14.7842 16.375 0.0345757 0.0676123 2026-02-25T08:43:38.338 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 122 16 28890 28874 14.7911 15.625 0.0609292 0.0675735 2026-02-25T08:43:38.504 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:38.584 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:38.585+0000 7f79bec35640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:38.586 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:38.588+0000 7f79bec35640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:38.617 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:38.751 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:38.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:39.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 123 16 29144 29128 14.7999 15.875 0.0684217 0.0675453 2026-02-25T08:43:40.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 124 16 29403 29387 14.8111 16.1875 0.0274298 0.0675004 2026-02-25T08:43:40.753 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:40.834 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:40.835+0000 7f4ef72cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:40.836 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:40.839+0000 7f4ef72cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:40.853 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:40.884 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:40.957 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:40.959 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:40.960 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:40.974 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:40.975 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:41.075 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~110s 2026-02-25T08:43:41.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 125 16 29652 29636 14.8171 15.5625 0.0515173 0.0674626 2026-02-25T08:43:42.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 126 16 29928 29912 14.8364 17.25 0.0647894 0.0673753 2026-02-25T08:43:42.976 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:43.058 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:43.059+0000 7f06cce11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:43.061 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:43.063+0000 7f06cce11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:43.077 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:43.197 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:43.197 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:43.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 127 16 30204 30188 14.8554 17.25 0.0550225 0.067297 2026-02-25T08:43:44.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 128 16 30477 30461 14.8726 17.0625 0.0540959 0.0672089 2026-02-25T08:43:45.198 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:45.279 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:45.280+0000 7f6a4c17e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:45.281 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:45.284+0000 7f6a4c17e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:45.307 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:45.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 129 16 30717 30701 14.8736 15 0.0845656 0.0672043 2026-02-25T08:43:45.398 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:45.399 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:45.961 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:46.033 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:46.036 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:46.037 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:46.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 130 16 30977 30961 14.8842 16.25 0.0450043 0.0671608 2026-02-25T08:43:46.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~116s 2026-02-25T08:43:47.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 131 16 31257 31241 14.9041 17.5 0.0954478 0.0670798 2026-02-25T08:43:47.400 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:47.479 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:47.480+0000 7f27d7e3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:47.481 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:47.483+0000 7f27d7e3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:47.498 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:47.605 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:47.605 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:48.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 132 16 31520 31504 14.9158 16.4375 0.068477 0.0670337 2026-02-25T08:43:49.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 133 16 31761 31745 14.9168 15.0625 0.0278568 0.0669963 2026-02-25T08:43:49.606 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:49.684 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:49.685+0000 7fe0338cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:49.687 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:49.689+0000 7fe0338cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:49.706 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:49.813 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:49.813 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:50.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 134 16 31998 31982 14.9161 14.8125 0.0641883 0.0670191 2026-02-25T08:43:51.038 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:51.109 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:51.112 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:51.113 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:51.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 135 16 32258 32242 14.9259 16.25 0.0462692 0.0669802 2026-02-25T08:43:51.814 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:51.893 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:51.894+0000 7f744883d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:51.895 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:51.897+0000 7f744883d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:51.920 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:51.999 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:52.000 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:52.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~121s 2026-02-25T08:43:52.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 136 16 32505 32489 14.9297 15.4375 0.0343899 0.0669492 2026-02-25T08:43:53.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 137 16 32748 32732 14.9316 15.1875 0.0546161 0.0669521 2026-02-25T08:43:54.001 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:54.081 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:54.082+0000 7f109483d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:54.084 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:54.086+0000 7f109483d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:54.113 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:54.187 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:54.188 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:54.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 138 16 32995 32979 14.9352 15.4375 0.0533964 0.0669362 2026-02-25T08:43:55.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 139 16 33257 33241 14.9456 16.375 0.0780397 0.0668936 2026-02-25T08:43:56.114 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:43:56.189 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:43:56.192 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:43:56.194 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:43:56.195 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:43:56.269 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:56.270+0000 7fa6884c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:56.272 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:56.274+0000 7fa6884c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:56.301 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:56.339 INFO:tasks.radosbench.radosbench.0.trial042.stdout:2026-02-25T08:43:56.343363+0000 min lat: 0.0107423 max lat: 24.1039 avg lat: 0.0668449 2026-02-25T08:43:56.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-25T08:43:56.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 140 16 33517 33501 14.9549 16.25 0.0304118 0.0668449 2026-02-25T08:43:56.455 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:56.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:57.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 141 16 33765 33749 14.9588 15.5 0.0413695 0.0668345 2026-02-25T08:43:57.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~127s 2026-02-25T08:43:58.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 142 16 34044 34028 14.9762 17.4375 0.0313712 0.0667553 2026-02-25T08:43:58.457 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:43:58.535 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:58.536+0000 7fcec8f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:58.537 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:43:58.540+0000 7fcec8f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:58.561 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:43:58.648 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:43:58.648 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:43:59.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 143 16 34294 34278 14.9807 15.625 0.094029 0.0667297 2026-02-25T08:44:00.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 144 16 34533 34517 14.9804 14.9375 0.169765 0.0667269 2026-02-25T08:44:00.649 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:00.731 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:00.732+0000 7fb79a70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:00.733 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:00.735+0000 7fb79a70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:00.757 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:00.835 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:00.836 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:01.196 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:01.268 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:01.271 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:01.272 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:01.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 145 16 34775 34759 14.9814 15.125 0.109229 0.0667281 2026-02-25T08:44:02.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 146 16 35026 35010 14.9862 15.6875 0.0300366 0.0667 2026-02-25T08:44:02.836 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:02.916 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:02.917+0000 7f494ba11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:02.918 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:02.921+0000 7f494ba11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:02.949 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:03.039 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:03.040 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:03.090 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~132s 2026-02-25T08:44:03.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 147 16 35269 35253 14.9876 15.1875 0.0464203 0.0667067 2026-02-25T08:44:04.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 148 16 35532 35516 14.9974 16.4375 0.0767051 0.0666566 2026-02-25T08:44:05.040 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:05.121 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:05.122+0000 7f1f4785a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:05.124 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:05.126+0000 7f1f4785a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:05.145 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:05.289 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:05.290 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:05.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 149 16 35790 35774 15.005 16.125 0.023812 0.0666169 2026-02-25T08:44:06.272 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:06.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout: 150 14 36041 36027 15.0103 15.8125 0.0353615 0.0666018 2026-02-25T08:44:06.347 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:06.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:06.350 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:07.290 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:07.340 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Total time run: 150.048 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Total writes made: 36041 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Write size: 65536 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Object size: 65536 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Bandwidth (MB/sec): 15.0122 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Stddev Bandwidth: 6.80174 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Max bandwidth (MB/sec): 23.125 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Min bandwidth (MB/sec): 0 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Average IOPS: 240 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Stddev IOPS: 108.832 2026-02-25T08:44:07.341 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Max IOPS: 370 2026-02-25T08:44:07.342 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Min IOPS: 0 2026-02-25T08:44:07.342 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Average Latency(s): 0.066604 2026-02-25T08:44:07.342 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Stddev Latency(s): 0.505563 2026-02-25T08:44:07.342 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Max latency(s): 24.1039 2026-02-25T08:44:07.342 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Min latency(s): 0.0107423 2026-02-25T08:44:07.362 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Cleaning up (deleting benchmark objects) 2026-02-25T08:44:07.369 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:07.371+0000 7ff3231f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:07.372 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:07.374+0000 7ff3231f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:07.399 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:07.494 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:07.495 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:08.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~138s 2026-02-25T08:44:09.496 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:09.574 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:09.575+0000 7f8a91374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:09.576 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:09.579+0000 7f8a91374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:09.623 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:09.785 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:09.786 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:11.351 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:11.425 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:11.427 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:11.428 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:11.786 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:11.866 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:11.868+0000 7f5cfbf0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:11.869 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:11.872+0000 7f5cfbf0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:11.897 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:12.006 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:12.006 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:14.007 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:14.084 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:14.086+0000 7f31d36cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:14.087 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:14.090+0000 7f31d36cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:14.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~143s 2026-02-25T08:44:14.116 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:14.246 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:14.246 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:16.247 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:16.327 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:16.328+0000 7f1b6665a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:16.330 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:16.333+0000 7f1b6665a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:16.351 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:16.430 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:16.503 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:16.504 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:16.504 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:16.505 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:16.506 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:18.505 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:18.582 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:18.584+0000 7f8b5bcc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:18.584 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:18.588+0000 7f8b5bcc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:18.618 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:18.757 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:18.757 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:19.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~149s 2026-02-25T08:44:20.759 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:20.837 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:20.839+0000 7f3e78d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:20.840 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:20.843+0000 7f3e78d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:20.876 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:21.018 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:21.019 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:21.507 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:21.578 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:21.580 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:21.581 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:23.020 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:23.100 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:23.101+0000 7f02309dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:23.102 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:23.105+0000 7f02309dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:23.145 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:23.272 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:23.273 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:25.105 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~154s 2026-02-25T08:44:25.274 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:25.354 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:25.355+0000 7f3a1e19c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:25.357 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:25.359+0000 7f3a1e19c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:25.385 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:25.503 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:25.504 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:26.582 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:26.656 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:26.658 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:26.659 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:27.504 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:27.581 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:27.583+0000 7f20bf10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:27.584 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:27.586+0000 7f20bf10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:27.610 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:27.758 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:27.759 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:29.761 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:29.839 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:29.840+0000 7f319c6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:29.842 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:29.844+0000 7f319c6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:29.877 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:29.997 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:29.997 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:30.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~160s 2026-02-25T08:44:31.660 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:31.734 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:31.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:31.737 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:31.998 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:32.077 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:32.078+0000 7fcc5ec11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:32.079 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:32.082+0000 7fcc5ec11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:32.106 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:32.183 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:32.183 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:34.185 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:34.267 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:34.268+0000 7fa5d830b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:34.270 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:34.272+0000 7fa5d830b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:34.286 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:34.398 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:34.399 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:36.112 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~165s 2026-02-25T08:44:36.400 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:36.482 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:36.483+0000 7f24785c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:36.485 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:36.487+0000 7f24785c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:36.501 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:36.659 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:36.660 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:36.738 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:36.813 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:36.816 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:36.817 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:38.661 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:38.740 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:38.741+0000 7fa7bc45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:38.746 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:38.748+0000 7fa7bc45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:38.780 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:38.862 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:38.862 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:40.863 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:40.942 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:40.943+0000 7f22cc99c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:40.944 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:40.947+0000 7f22cc99c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:40.971 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:41.085 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:41.086 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:41.617 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~171s 2026-02-25T08:44:41.817 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:41.891 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:41.894 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:41.894 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:43.087 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:43.167 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:43.168+0000 7f0f3ef0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:43.169 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:43.172+0000 7f0f3ef0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:43.189 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:43.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:43.245 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:45.246 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:45.329 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:45.330+0000 7f2133b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:45.331 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:45.334+0000 7f2133b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:45.378 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:45.521 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:45.521 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:46.895 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:46.968 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:46.972 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:46.972 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:47.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~176s 2026-02-25T08:44:47.522 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:47.602 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:47.603+0000 7f4c3e764640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:47.605 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:47.607+0000 7f4c3e764640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:47.632 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:47.775 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:47.776 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:49.777 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:49.857 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:49.857+0000 7f8e01374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:49.860 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:49.862+0000 7f8e01374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:49.881 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:49.999 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:50.000 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:51.973 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:52.001 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:52.046 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:52.049 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:52.049 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:52.092 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:52.093+0000 7f54c603d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:52.095 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:52.097+0000 7f54c603d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:52.113 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:52.181 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:52.181 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:52.626 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~182s 2026-02-25T08:44:54.182 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:54.263 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:54.264+0000 7f25ee45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:54.266 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:54.269+0000 7f25ee45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:54.292 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:54.422 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:54.423 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:56.424 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:44:56.505 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:56.507+0000 7fef41635640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:56.507 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:56.510+0000 7fef41635640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:56.528 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:56.670 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:56.671 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:44:57.051 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:44:57.123 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:44:57.125 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:44:57.126 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:44:58.130 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~187s 2026-02-25T08:44:58.672 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:44:58.754 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:58.756+0000 7f193c25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:58.756 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:44:58.759+0000 7f193c25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:44:58.782 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:44:58.885 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:44:58.886 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:00.887 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:00.968 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:00.970+0000 7face6e1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:00.970 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:00.974+0000 7face6e1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:00.991 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:01.150 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:01.151 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:02.127 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:02.198 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:02.200 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:02.201 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:03.153 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:03.230 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:03.232+0000 7f7dc3d8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:03.232 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:03.236+0000 7f7dc3d8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:03.295 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:03.427 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:03.427 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:03.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~193s 2026-02-25T08:45:05.428 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:05.507 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:05.510+0000 7fbc753c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:05.510 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:05.514+0000 7fbc753c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:05.547 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:05.723 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:05.723 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:07.202 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:07.272 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:07.275 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:07.276 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:07.724 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:07.804 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:07.806+0000 7f075c50b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:07.807 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:07.810+0000 7f075c50b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:07.862 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:07.981 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:07.982 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:09.139 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~198s 2026-02-25T08:45:09.983 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:10.063 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:10.065+0000 7f4e2bd5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:10.066 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:10.068+0000 7f4e2bd5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:10.094 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:10.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:10.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:12.245 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:12.276 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:12.326 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:12.328+0000 7f052efdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:12.329 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:12.332+0000 7f052efdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:12.352 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:12.361 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:12.364 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:12.365 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:12.480 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:12.481 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:14.482 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:14.562 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:14.562+0000 7f90bc05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:14.565 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:14.567+0000 7f90bc05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:14.596 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:14.643 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~204s 2026-02-25T08:45:14.735 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:14.736 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:16.736 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:16.814 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:16.815+0000 7f400988b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:16.818 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:16.821+0000 7f400988b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:16.848 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:17.024 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:17.025 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:17.365 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:17.439 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:17.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:17.442 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:19.026 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:19.106 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:19.108+0000 7f2825bdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:19.109 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:19.111+0000 7f2825bdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:19.140 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:19.273 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:19.274 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:20.147 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~209s 2026-02-25T08:45:21.275 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:21.353 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:21.354+0000 7f3a18acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:21.355 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:21.357+0000 7f3a18acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:21.385 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:21.588 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:21.588 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:22.443 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:22.517 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:22.520 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:22.521 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:23.589 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:23.668 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:23.669+0000 7f4b08764640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:23.670 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:23.673+0000 7f4b08764640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:23.685 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:23.837 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:23.837 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:25.651 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~215s 2026-02-25T08:45:25.839 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:25.917 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:25.918+0000 7f511a1c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:25.920 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:25.922+0000 7f511a1c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:25.935 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:26.047 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:26.047 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:27.522 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:27.594 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:27.597 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:27.597 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:28.049 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:28.127 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:28.128+0000 7fdb316cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:28.130 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:28.133+0000 7fdb316cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:28.170 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:28.389 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:28.390 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:30.390 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:30.470 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:30.471+0000 7fb5eee5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:30.473 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:30.475+0000 7fb5eee5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:30.494 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:30.571 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:30.571 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:31.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~220s 2026-02-25T08:45:32.572 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:32.599 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:32.651 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:32.652+0000 7fa7e33a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:32.654 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:32.656+0000 7fa7e33a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:32.685 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:32.688 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:32.691 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:32.692 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:32.833 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:32.834 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:34.835 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:34.912 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:34.913+0000 7f1e190c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:34.914 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:34.916+0000 7f1e190c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:34.950 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:35.052 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:35.052 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:36.658 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~226s 2026-02-25T08:45:37.053 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:37.134 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:37.135+0000 7f8473235640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:37.137 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:37.139+0000 7f8473235640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:37.155 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:37.340 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:37.341 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:37.693 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:37.767 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:37.770 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:37.770 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:39.343 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:39.420 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:39.421+0000 7f179e25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:39.423 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:39.425+0000 7f179e25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:39.466 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:39.559 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:39.560 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:41.561 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:41.639 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:41.641+0000 7fe328dd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:41.642 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:41.645+0000 7fe328dd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:41.664 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:41.822 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:41.823 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:42.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~231s 2026-02-25T08:45:42.772 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:42.846 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:42.849 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:42.849 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:43.824 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:43.901 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:43.903+0000 7f8a44acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:43.904 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:43.907+0000 7f8a44acb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:43.930 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:44.061 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:44.061 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:46.062 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:46.142 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:46.144+0000 7f038541a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:46.145 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:46.149+0000 7f038541a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:46.167 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:46.333 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:46.334 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:47.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~237s 2026-02-25T08:45:47.851 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:47.921 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:47.924 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:47.925 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:48.335 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:48.414 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:48.416+0000 7fb23030b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:48.416 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:48.420+0000 7fb23030b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:48.455 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:48.661 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:48.662 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:50.663 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:50.743 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:50.745+0000 7ff7fdf0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:50.746 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:50.749+0000 7ff7fdf0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:50.765 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:50.931 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:50.932 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:52.926 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:52.933 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:53.000 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:53.002 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:53.003 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:53.048 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:53.050+0000 7f3f3ef0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:53.050 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:53.054+0000 7f3f3ef0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:53.066 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:53.171 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~242s 2026-02-25T08:45:53.175 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:53.175 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:55.176 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:55.255 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:55.256+0000 7f0a42d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:55.257 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:55.260+0000 7f0a42d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:55.284 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:55.354 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:55.355 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:57.356 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:45:57.434 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:57.435+0000 7f561c446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:57.436 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:57.439+0000 7f561c446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:57.464 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:57.654 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:57.655 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:45:58.003 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:45:58.076 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:45:58.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:45:58.080 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:45:58.675 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~248s 2026-02-25T08:45:59.655 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:45:59.732 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:59.734+0000 7f544e964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:59.735 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:45:59.737+0000 7f544e964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:45:59.756 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:45:59.919 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:45:59.919 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:01.920 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:01.998 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:02.000+0000 7f4fb5a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:02.000 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:02.003+0000 7f4fb5a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:02.044 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:02.128 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:02.129 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:03.081 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:03.155 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:03.157 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:03.158 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:04.130 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:04.178 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~253s 2026-02-25T08:46:04.208 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:04.209+0000 7f4bb06cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:04.210 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:04.213+0000 7f4bb06cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:04.241 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:04.396 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:04.397 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:06.398 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:06.474 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:06.475+0000 7f5276bdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:06.476 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:06.478+0000 7f5276bdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:06.494 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:06.654 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:06.655 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:08.159 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:08.231 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:08.234 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:08.235 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:08.656 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:08.734 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:08.736+0000 7fb37721a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:08.737 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:08.739+0000 7fb37721a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:08.753 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:08.884 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:08.885 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:09.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~259s 2026-02-25T08:46:10.886 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:10.965 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:10.966+0000 7fabf8406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:10.967 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:10.969+0000 7fabf8406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:10.984 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:11.030 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:11.030 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:13.032 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:13.108 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:13.109+0000 7f134cb9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:13.110 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:13.113+0000 7f134cb9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:13.138 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:13.230 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:13.230 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:13.236 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:13.309 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:13.311 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:13.312 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:15.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~264s 2026-02-25T08:46:15.231 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:15.311 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:15.312+0000 7f6f22b5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:15.313 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:15.316+0000 7f6f22b5b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:15.350 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:15.463 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:15.463 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:17.464 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:17.540 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:17.542+0000 7f500bbe4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:17.543 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:17.545+0000 7f500bbe4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:17.568 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:17.751 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:17.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:18.313 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:18.385 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:18.388 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:18.388 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:19.753 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:19.829 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:19.830+0000 7f7d0845a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:19.831 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:19.834+0000 7f7d0845a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:19.877 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:20.045 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:20.045 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:20.690 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~270s 2026-02-25T08:46:22.047 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:22.128 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:22.129+0000 7f4dc890b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:22.130 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:22.133+0000 7f4dc890b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:22.159 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:22.252 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:22.253 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:23.389 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:23.464 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:23.467 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:23.467 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:24.254 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:24.333 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:24.335+0000 7fe41cf9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:24.336 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:24.339+0000 7fe41cf9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:24.360 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:24.534 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:24.535 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:26.194 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~275s 2026-02-25T08:46:26.536 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:26.614 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:26.615+0000 7fa3bd8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:26.617 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:26.619+0000 7fa3bd8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:26.639 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:26.836 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:26.837 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:28.468 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:28.542 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:28.545 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:28.546 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:28.837 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:28.920 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:28.922+0000 7f5f8dacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:28.922 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:28.926+0000 7f5f8dacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:28.947 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:29.047 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:29.048 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:31.049 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:31.129 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:31.131+0000 7fad44ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:31.131 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:31.135+0000 7fad44ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:31.158 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:31.255 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:31.256 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:31.699 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~281s 2026-02-25T08:46:33.257 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:33.334 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:33.336+0000 7f64d405a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:33.336 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:33.340+0000 7f64d405a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:33.356 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:33.440 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:33.441 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:33.546 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:33.620 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:33.623 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:33.624 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:35.441 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:35.524 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:35.526+0000 7fb89becb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:35.527 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:35.530+0000 7fb89becb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:35.544 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:35.740 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:35.740 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:37.201 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~286s 2026-02-25T08:46:37.741 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:37.823 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:37.825+0000 7f5d7183d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:37.825 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:37.829+0000 7f5d7183d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:37.861 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:37.953 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:37.954 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:38.624 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:38.698 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:38.701 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:38.702 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:39.954 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:40.035 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:40.038+0000 7f21016c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:40.038 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:40.042+0000 7f21016c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:40.057 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:40.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:40.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:42.255 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:42.335 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:42.336+0000 7feb82a1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:42.338 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:42.340+0000 7feb82a1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:42.362 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:42.471 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:42.471 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:42.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~292s 2026-02-25T08:46:43.703 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:43.775 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:43.778 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:43.779 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:44.472 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:44.555 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:44.556+0000 7f414428b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:44.558 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:44.560+0000 7f414428b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:44.574 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:44.741 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:44.742 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:46.407 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Removed 36041 objects 2026-02-25T08:46:46.407 INFO:tasks.radosbench.radosbench.0.trial042.stdout:Clean up completed and total clean up time :159.044 2026-02-25T08:46:46.743 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2026-02-25T08:46:46.821 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:46.822+0000 7f69025ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:46.823 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:46.826+0000 7f69025ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:46.839 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:46.866 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:46.866 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:48.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~297s 2026-02-25T08:46:48.780 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:48.856 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:48.858 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:48.859 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:48.867 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2026-02-25T08:46:48.895 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2026-02-25T08:46:48.895 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2026-02-25T08:46:48.942 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:48.943+0000 7f357e2b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:48.944 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:48.947+0000 7f357e2b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:48.960 INFO:teuthology.orchestra.run.trial042.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-25T08:46:48.986 DEBUG:teuthology.orchestra.run:got remote process result: 6 2026-02-25T08:46:48.987 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2026-02-25T08:46:48.988 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:48.986+0000 7fe12e90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:48.991 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:46:48.993+0000 7fe12e90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:46:49.264 INFO:teuthology.orchestra.run.trial042.stderr:pool 'unique_pool_0' removed 2026-02-25T08:46:49.276 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2026-02-25T08:46:49.292 INFO:tasks.thrashosds:joining thrashosds 2026-02-25T08:46:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~303s 2026-02-25T08:46:53.713 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2026-02-25T08:46:53.860 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:53.931 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:53.934 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:53.935 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:46:54.217 INFO:tasks.ceph.osd.0:Sent signal 15 2026-02-25T08:46:54.217 INFO:tasks.ceph.osd.2:Sent signal 15 2026-02-25T08:46:54.218 INFO:tasks.ceph.osd.3:Sent signal 15 2026-02-25T08:46:54.218 INFO:tasks.ceph.mon.a:Sent signal 15 2026-02-25T08:46:54.218 INFO:tasks.ceph.mgr.x:Sent signal 15 2026-02-25T08:46:54.218 INFO:tasks.daemonwatchdog.daemon_watchdog:Killing thrasher OSDThrasher 2026-02-25T08:46:54.219 INFO:tasks.ceph.mon.a.trial042.stderr:2026-02-25T08:46:54.221+0000 7f89d3bfc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 40094) UID: 0 2026-02-25T08:46:54.219 INFO:tasks.ceph.mon.a.trial042.stderr:2026-02-25T08:46:54.221+0000 7f89d3bfc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-25T08:46:54.229 INFO:tasks.ceph.mgr.x.trial066.stderr:2026-02-25T08:46:54.226+0000 7f4c9b5f8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 39668) UID: 0 2026-02-25T08:46:54.229 INFO:tasks.ceph.mgr.x.trial066.stderr:2026-02-25T08:46:54.226+0000 7f4c9b5f8640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-25T08:46:58.936 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:46:59.008 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:46:59.011 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:46:59.011 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:04.012 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:04.084 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:04.087 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:04.088 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:09.089 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:09.162 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:09.165 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:09.166 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:14.167 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:14.239 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:14.242 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:14.242 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:19.243 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:19.313 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:19.315 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:19.316 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:24.317 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:24.388 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:24.391 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:24.391 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:29.392 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:29.463 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:29.466 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:29.467 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:34.468 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:34.540 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:34.543 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:34.543 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:39.545 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:39.617 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:39.620 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:39.621 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:44.622 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:44.693 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:44.696 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:44.697 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2026-02-25T08:47:49.698 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2026-02-25T08:47:49.768 INFO:teuthology.orchestra.run.trial042.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2026-02-25T08:47:49.771 DEBUG:teuthology.orchestra.run:got remote process result: 22 2026-02-25T08:47:49.776 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 192, in wrapper return func(self) ^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1513, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 562, in revive_osd self.ceph_manager.revive_osd( File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 3236, in revive_osd self.wait_run_admin_socket('osd', osd, File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2085, in wait_run_admin_socket raise Exception('timed out waiting for admin_socket ' Exception: timed out waiting for admin_socket to appear after osd.1 restart 2026-02-25T08:47:49.776 ERROR:tasks.thrashosds.thrasher:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1384, in do_thrash self._do_thrash() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 192, in wrapper return func(self) ^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1513, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 562, in revive_osd self.ceph_manager.revive_osd( File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 3236, in revive_osd self.wait_run_admin_socket('osd', osd, File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2085, in wait_run_admin_socket raise Exception('timed out waiting for admin_socket ' Exception: timed out waiting for admin_socket to appear after osd.1 restart 2026-02-25T08:47:49.778 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2026-02-25T08:47:49.778 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-25T08:47:49.778 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:47:49.779 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2026-02-25T08:47:49.855 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:47:49.855+0000 7fc83e5a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:49:49.810 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-25T08:49:49.811 ERROR:teuthology.run_tasks:Manager failed: thrashosds Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 218, in task cluster_manager.wait_for_all_osds_up() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2970, in wait_for_all_osds_up while not self.are_all_osds_up(): ^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2960, in are_all_osds_up x = self.get_osd_dump() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2670, in get_osd_dump return self.get_osd_dump_json()['osds'] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2662, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2026-02-25T08:49:49.815 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-25T08:49:49.832 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T08:49:49.832 DEBUG:teuthology.orchestra.run.trial042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:49:49.909 INFO:teuthology.orchestra.run.trial042.stderr:2026-02-25T08:49:49.909+0000 7fb97685a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:51:49.864 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-25T08:51:49.865 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 218, in task cluster_manager.wait_for_all_osds_up() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2970, in wait_for_all_osds_up while not self.are_all_osds_up(): ^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2960, in are_all_osds_up x = self.get_osd_dump() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2670, in get_osd_dump return self.get_osd_dump_json()['osds'] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2662, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-25T08:51:49.866 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-25T08:51:49.866 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-25T08:51:49.866 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-25T08:51:49.866 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:49.927 INFO:tasks.ceph.osd.0:Stopped 2026-02-25T08:51:49.927 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-25T08:51:49.927 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:49.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:51:49.928 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 218, in task cluster_manager.wait_for_all_osds_up() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2970, in wait_for_all_osds_up while not self.are_all_osds_up(): ^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2960, in are_all_osds_up x = self.get_osd_dump() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2670, in get_osd_dump return self.get_osd_dump_json()['osds'] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2662, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 1525, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log' 2026-02-25T08:51:49.930 INFO:tasks.ceph.osd.1:Stopped 2026-02-25T08:51:49.930 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-25T08:51:49.930 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:50.020 INFO:tasks.ceph.osd.2:Stopped 2026-02-25T08:51:50.021 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-25T08:51:50.021 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:50.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:51:50.021 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 218, in task cluster_manager.wait_for_all_osds_up() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2970, in wait_for_all_osds_up while not self.are_all_osds_up(): ^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2960, in are_all_osds_up x = self.get_osd_dump() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2670, in get_osd_dump return self.get_osd_dump_json()['osds'] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2662, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 1525, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial066 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log' 2026-02-25T08:51:50.026 INFO:tasks.ceph.osd.3:Stopped 2026-02-25T08:51:50.027 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-25T08:51:50.027 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-25T08:51:50.027 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:50.027 INFO:tasks.ceph.mgr.x:Stopped 2026-02-25T08:51:50.027 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-25T08:51:50.027 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-25T08:51:50.028 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:51:50.028 INFO:tasks.ceph.mon.a:Stopped 2026-02-25T08:51:50.028 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-25T08:51:50.029 DEBUG:teuthology.orchestra.run.trial042:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v 'but it is still running' | grep -E -v 'objects unfound and apparently lost' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(OSD_' | grep -E -v '\(PG_' | grep -E -v '\(POOL_' | grep -E -v '\(CACHE_POOL_' | grep -E -v '\(SMALLER_PGP_NUM\)' | grep -E -v '\(OBJECT_' | grep -E -v SLOW_OPS | grep -E -v '\(REQUEST_SLOW\)' | grep -E -v '\(TOO_FEW_PGS\)' | grep -E -v 'slow request' | grep -E -v 'timeout on replica' | grep -E -v 'late reservation from' | grep -E -v MON_DOWN | grep -E -v OSDMAP_FLAGS | grep -E -v OSD_DOWN | grep -E -v PG_DEGRADED | grep -E -v PG_AVAILABILITY | grep -E -v POOL_APP_NOT_ENABLED | grep -E -v 'mons down' | grep -E -v 'mon down' | grep -E -v 'out of quorum' | grep -E -v noscrub | grep -E -v nodeep-scrub | grep -E -v 'Degraded data redundancy' | grep -E -v 'is down' | grep -E -v 'osds down' | grep -E -v 'pg .*? is .*?degraded.*?, acting' | grep -E -v 'pg .*? is stuck' | grep -E -v 'pg degraded' | grep -E -v PG_BACKFILL_FULL | grep -E -v 'Low space hindering backfill .*? backfill_toofull' | grep -E -v OSD_ROOT_DOWN | grep -E -v 'pgs degraded' | grep -E -v 'pgs undersized' | grep -E -v 'is active.*backfill_toofull.*' | head -n 1 2026-02-25T08:51:50.059 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial042.front.sepia.ceph.com 2026-02-25T08:51:50.059 DEBUG:teuthology.orchestra.run.trial042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-25T08:51:50.126 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial042.front.sepia.ceph.com 2026-02-25T08:51:50.126 DEBUG:teuthology.orchestra.run.trial042:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-25T08:51:50.192 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial066.front.sepia.ceph.com 2026-02-25T08:51:50.192 DEBUG:teuthology.orchestra.run.trial066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-25T08:51:50.226 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial066.front.sepia.ceph.com 2026-02-25T08:51:50.227 DEBUG:teuthology.orchestra.run.trial066:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-25T08:51:50.296 INFO:tasks.ceph:Archiving mon data... 2026-02-25T08:51:50.298 DEBUG:teuthology.misc:Transferring archived files from trial042:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053/data/mon.a.tgz 2026-02-25T08:51:50.299 DEBUG:teuthology.orchestra.run.trial042:> mktemp 2026-02-25T08:51:50.318 INFO:teuthology.orchestra.run.trial042.stdout:/tmp/tmp.uS6kJJ8Cqv 2026-02-25T08:51:50.319 DEBUG:teuthology.orchestra.run.trial042:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uS6kJJ8Cqv 2026-02-25T08:51:50.414 DEBUG:teuthology.orchestra.run.trial042:> sudo chmod 0666 /tmp/tmp.uS6kJJ8Cqv 2026-02-25T08:51:50.497 DEBUG:teuthology.orchestra.remote:trial042:/tmp/tmp.uS6kJJ8Cqv is 272KB 2026-02-25T08:51:50.594 DEBUG:teuthology.orchestra.run.trial042:> rm -fr /tmp/tmp.uS6kJJ8Cqv 2026-02-25T08:51:50.610 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-25T08:51:50.611 DEBUG:teuthology.orchestra.run.trial042:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T08:51:50.651 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T08:51:50.730 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T08:51:50.746 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T08:51:50.746 INFO:tasks.ceph:Archiving crash dumps... 2026-02-25T08:51:50.749 DEBUG:teuthology.misc:Transferring archived files from trial042:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053/remote/trial042/crash 2026-02-25T08:51:50.750 DEBUG:teuthology.orchestra.run.trial042:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T08:51:50.794 DEBUG:teuthology.misc:Transferring archived files from trial066:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053/remote/trial066/crash 2026-02-25T08:51:50.795 DEBUG:teuthology.orchestra.run.trial066:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T08:51:50.820 INFO:tasks.ceph:Compressing logs... 2026-02-25T08:51:50.820 DEBUG:teuthology.orchestra.run.trial042:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:51:50.839 DEBUG:teuthology.orchestra.run.trial066:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:51:50.893 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-25T08:51:50.893 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39768.log 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39866.log 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr:/var/log/ceph/ceph-client.admin.39768.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39768.log.gz 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39920.log 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39815.log 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr:/var/log/ceph/ceph-client.admin.39866.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39866.log.gz 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr:/var/log/ceph/ceph-client.admin.39920.log: /var/log/ceph/ceph-client.admin.39815.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39920.log.gz 2026-02-25T08:51:50.894 INFO:teuthology.orchestra.run.trial066.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39815.log.gz 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46326.log 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44078.log 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40742.log 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46326.log.gz 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44078.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44078.log.gz 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50214.log 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40742.log: /var/log/ceph/ceph-client.admin.50214.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50214.log.gz 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40742.log.gz 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45407.log 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41018.log 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43346.log 2026-02-25T08:51:50.908 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45407.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45461.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41018.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43346.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45461.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45461.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43019.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41410.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.43019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46515.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.43019.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41410.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41410.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41356.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41581.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46515.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46515.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41356.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41356.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40676.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41771.log 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41581.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40676.log: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.40676.log.gz 2026-02-25T08:51:50.909 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47514.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41771.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47568.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50539.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40525.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47514.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50539.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50539.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41016.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41215.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40525.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41016.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41016.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51656.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46839.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41215.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.51656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49755.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr: 8.0% -- replaced with /var/log/ceph/ceph-client.admin.51656.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46839.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46839.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49080.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50403.log 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49755.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49080.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49080.log.gz 2026-02-25T08:51:50.910 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45272.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50403.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45272.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45272.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47649.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51215.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47649.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47649.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46974.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.51215.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.51215.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51242.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46974.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46974.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47379.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.51242.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51242.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44730.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47379.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47379.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44486.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44730.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44730.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44349.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44486.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44486.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51080.log 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44349.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44349.log.gz 2026-02-25T08:51:50.911 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.51080.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.51080.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49890.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44405.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44405.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50864.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49890.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49890.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50864.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50864.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41627.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43562.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41357.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41627.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41627.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47433.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41357.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41357.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47433.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47433.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48837.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44676.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48837.log: gzip -5 49.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.48837.log.gz -- 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr: /var/log/ceph/ceph-client.admin.43101.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44676.log: 50.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44676.log.gz 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47109.log 2026-02-25T08:51:50.912 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43101.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43101.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41722.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47109.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47109.log.gz -5 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41722.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41722.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50620.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48702.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48243.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50620.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50620.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46191.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48243.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48243.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45326.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46191.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46191.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48891.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.45326.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48756.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43427.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48891.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48891.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47244.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43427.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43427.log.gz 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48297.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49026.log 2026-02-25T08:51:50.913 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47244.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48297.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48297.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45596.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49026.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49026.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45000.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45596.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45596.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50025.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45000.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45000.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45921.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50025.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50025.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49485.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45921.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49485.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49485.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41869.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45731.log: /var/log/ceph/ceph-client.admin.41580.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41869.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41869.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45137.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50133.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41580.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49809.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45137.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41526.log 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50133.log.gz 2026-02-25T08:51:50.914 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48162.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49809.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45191.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41526.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45081.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48162.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44946.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45191.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44023.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.45081.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44946.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44946.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49620.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43941.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44023.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49620.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49620.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44133.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43941.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43941.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42684.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40840.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44133.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42684.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42684.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49296.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40840.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45867.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45650.log 2026-02-25T08:51:50.915 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40651.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49296.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45650.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51134.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40651.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41017.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43292.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.51134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46920.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.51134.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43292.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43292.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42740.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46920.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47838.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42740.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42740.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46137.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47838.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47838.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43156.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46137.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46137.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46380.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43156.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43156.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48621.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46380.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46380.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46650.log 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48621.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48621.log.gz 2026-02-25T08:51:50.916 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46596.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40097.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46650.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48945.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40097.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40097.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44595.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48945.log: gzip -5 --verbose -- 50.4% /var/log/ceph/ceph.audit.log -- replaced with /var/log/ceph/ceph-client.admin.48945.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr: 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42883.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44595.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41188.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42883.log: 87.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47784.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42883.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43211.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41188.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47784.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47784.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43670.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49674.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43211.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43211.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43670.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43670.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50349.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49674.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50349.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50349.log.gz 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48486.log 2026-02-25T08:51:50.917 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43481.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48486.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48486.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49566.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49944.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43481.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49566.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49566.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48108.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49944.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49944.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48108.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48108.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44268.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45785.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44268.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44268.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46002.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47892.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.45785.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46002.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46002.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40889.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40791.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47892.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40889.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40889.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47190.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40791.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40791.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50268.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40581.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40183.log 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47190.log.gz 2026-02-25T08:51:50.918 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40581.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40581.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50484.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50079.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40183.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41187.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50484.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50079.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50079.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44811.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50999.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41187.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44811.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44811.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46461.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44865.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50999.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46461.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46461.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47703.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41411.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44865.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47703.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47703.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40438.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42482.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41411.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40438.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40438.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43752.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41186.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.42482.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43752.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43752.log.gz 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47298.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48027.log 2026-02-25T08:51:50.919 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41186.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47298.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47298.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48567.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.48027.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42016.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41967.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41019.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48567.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41967.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41967.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46245.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48432.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41019.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46245.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46245.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46704.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48432.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48432.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42191.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46704.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46704.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50674.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42191.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42191.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.39033.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50674.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50674.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44214.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph.tmp-client.admin.39033.log: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.39033.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40464.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44214.log: gzip 50.4% -- replaced with /var/log/ceph/ceph-client.admin.44214.log.gz -5 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47973.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.40464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47055.log 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40464.log.gz 2026-02-25T08:51:50.920 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47973.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47973.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49431.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.47055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42290.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47055.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49431.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.49431.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49350.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.0.42290.log: 65.3% -- replaced with /var/log/ceph/ceph-client.0.42290.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44540.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49350.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49350.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42192.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.44540.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44540.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42192.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.42192.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42267.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43887.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42267.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42267.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49161.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr: 89.0% -- replaced with /var/log/ceph/ceph.log.gz 50.0% 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43887.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.48351.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50729.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42825.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49161.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43616.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50729.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.42825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.42825.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.43616.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.43616.log.gz 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50810.log 2026-02-25T08:51:50.921 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.49215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45542.log 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50810.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.50810.log.gz 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.45542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51693.log 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45542.log.gz 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.46785.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41820.log 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.51693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50945.log 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr: 7.2% -- replaced with /var/log/ceph/ceph-client.admin.51693.log.gz 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.41820.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41820.log.gz 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr:/var/log/ceph/ceph-client.admin.50945.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.50945.log.gz 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial042.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-25T08:51:50.922 INFO:teuthology.orchestra.run.trial066.stderr:/var/log/ceph/ceph-mgr.x.log: 92.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-25T08:51:50.958 INFO:teuthology.orchestra.run.trial042.stderr: 89.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-25T08:52:04.699 INFO:teuthology.orchestra.run.trial042.stderr: 90.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-25T08:52:04.701 INFO:teuthology.orchestra.run.trial042.stderr: 2026-02-25T08:52:04.701 INFO:teuthology.orchestra.run.trial042.stderr:real 0m13.849s 2026-02-25T08:52:04.701 INFO:teuthology.orchestra.run.trial042.stderr:user 0m13.437s 2026-02-25T08:52:04.701 INFO:teuthology.orchestra.run.trial042.stderr:sys 0m0.612s 2026-02-25T08:52:14.214 INFO:teuthology.orchestra.run.trial066.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-25T08:52:17.715 INFO:teuthology.orchestra.run.trial066.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-25T08:52:17.717 INFO:teuthology.orchestra.run.trial066.stderr: 2026-02-25T08:52:17.717 INFO:teuthology.orchestra.run.trial066.stderr:real 0m26.835s 2026-02-25T08:52:17.717 INFO:teuthology.orchestra.run.trial066.stderr:user 0m48.170s 2026-02-25T08:52:17.718 INFO:teuthology.orchestra.run.trial066.stderr:sys 0m1.925s 2026-02-25T08:52:17.718 INFO:tasks.ceph:Archiving logs... 2026-02-25T08:52:17.719 DEBUG:teuthology.misc:Transferring archived files from trial042:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053/remote/trial042/log 2026-02-25T08:52:17.720 DEBUG:teuthology.orchestra.run.trial042:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T08:52:19.027 DEBUG:teuthology.misc:Transferring archived files from trial066:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053/remote/trial066/log 2026-02-25T08:52:19.028 DEBUG:teuthology.orchestra.run.trial066:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T08:52:23.340 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 218, in task cluster_manager.wait_for_all_osds_up() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2970, in wait_for_all_osds_up while not self.are_all_osds_up(): ^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2960, in are_all_osds_up x = self.get_osd_dump() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2670, in get_osd_dump return self.get_osd_dump_json()['osds'] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2662, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-25T08:52:23.342 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T08:52:23.353 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/install/__init__.py", line 644, in task yield File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph.py", line 2000, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 218, in task cluster_manager.wait_for_all_osds_up() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2970, in wait_for_all_osds_up while not self.are_all_osds_up(): ^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2960, in are_all_osds_up x = self.get_osd_dump() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2670, in get_osd_dump return self.get_osd_dump_json()['osds'] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2662, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2026-02-25T08:52:23.486 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T08:52:23.500 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/install/__init__.py", line 644, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/thrashosds.py", line 218, in task cluster_manager.wait_for_all_osds_up() File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2970, in wait_for_all_osds_up while not self.are_all_osds_up(): ^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2960, in are_all_osds_up x = self.get_osd_dump() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2670, in get_osd_dump return self.get_osd_dump_json()['osds'] ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 2662, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2026-02-25T08:52:23.501 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-25T08:52:23.501 DEBUG:teuthology.orchestra.run.trial042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T08:52:23.503 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T08:52:23.529 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-25T08:52:23.540 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-25T08:52:23.540 DEBUG:teuthology.orchestra.run.trial042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:52:23.569 DEBUG:teuthology.orchestra.run.trial066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:52:23.582 INFO:teuthology.orchestra.run.trial042.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:52:23.584 INFO:teuthology.orchestra.run.trial066.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial066.stdout:=============================================================================== 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial066.stdout:^* time.cloudflare.com 3 8 377 241 -714us[ -877us] +/- 6975us 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial066.stdout:^- 72-14-182-49.ip.linodeus> 2 8 377 433 +3556us[+3403us] +/- 82ms 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial066.stdout:^- time1.weber.edu 1 8 377 234 -212us[ -212us] +/- 26ms 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial066.stdout:^- t2.time.gq1.yahoo.com 2 7 377 46 -38ms[ -38ms] +/- 70ms 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial042.stdout:=============================================================================== 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial042.stdout:^+ gwallaceNTPpool.com 2 8 377 242 +4687us[+4739us] +/- 65ms 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial042.stdout:^+ 72-14-182-49.ip.linodeus> 2 8 377 110 +9966us[+9993us] +/- 58ms 2026-02-25T08:52:23.671 INFO:teuthology.orchestra.run.trial042.stdout:^* time1.weber.edu 1 7 377 49 +5293us[+5321us] +/- 26ms 2026-02-25T08:52:23.672 INFO:teuthology.orchestra.run.trial042.stdout:^+ t2.time.gq1.yahoo.com 2 7 377 47 -32ms[ -32ms] +/- 70ms 2026-02-25T08:52:23.672 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-25T08:52:23.682 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-25T08:52:23.683 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-25T08:52:23.696 DEBUG:teuthology.orchestra.run.trial042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T08:52:23.766 DEBUG:teuthology.orchestra.run.trial066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T08:52:23.821 DEBUG:teuthology.orchestra.run.trial042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:52:23.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:52:23.849 DEBUG:teuthology.orchestra.run.trial066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:52:23.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:52:23.878 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-25T08:52:23.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-25T08:52:23.911 INFO:teuthology.task.internal:Duration was 2789.856677 seconds 2026-02-25T08:52:23.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-25T08:52:23.929 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-25T08:52:23.929 DEBUG:teuthology.orchestra.run.trial042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T08:52:23.932 DEBUG:teuthology.orchestra.run.trial066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T08:52:23.967 INFO:teuthology.orchestra.run.trial042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:52:23.969 INFO:teuthology.orchestra.run.trial066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:52:24.266 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-25T08:52:24.266 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial042.front.sepia.ceph.com 2026-02-25T08:52:24.267 DEBUG:teuthology.orchestra.run.trial042:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T08:52:24.286 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial066.front.sepia.ceph.com 2026-02-25T08:52:24.286 DEBUG:teuthology.orchestra.run.trial066:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T08:52:24.325 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-25T08:52:24.325 DEBUG:teuthology.orchestra.run.trial042:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:52:24.327 DEBUG:teuthology.orchestra.run.trial066:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:52:24.427 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-25T08:52:24.427 DEBUG:teuthology.orchestra.run.trial042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:52:24.429 DEBUG:teuthology.orchestra.run.trial066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:52:24.450 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:52:24.450 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:52:24.451 INFO:teuthology.orchestra.run.trial042.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T08:52:24.451 INFO:teuthology.orchestra.run.trial042.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:52:24.451 INFO:teuthology.orchestra.run.trial042.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T08:52:24.452 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:52:24.453 INFO:teuthology.orchestra.run.trial066.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:52:24.453 INFO:teuthology.orchestra.run.trial066.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:52:24.454 INFO:teuthology.orchestra.run.trial066.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T08:52:24.454 INFO:teuthology.orchestra.run.trial066.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T08:52:24.461 INFO:teuthology.orchestra.run.trial042.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T08:52:24.464 INFO:teuthology.orchestra.run.trial066.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T08:52:24.466 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-25T08:52:24.480 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-25T08:52:24.481 DEBUG:teuthology.orchestra.run.trial042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T08:52:24.526 DEBUG:teuthology.orchestra.run.trial066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T08:52:24.551 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-25T08:52:24.562 DEBUG:teuthology.orchestra.run.trial042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:52:24.568 DEBUG:teuthology.orchestra.run.trial066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:52:24.589 INFO:teuthology.orchestra.run.trial042.stdout:kernel.core_pattern = core 2026-02-25T08:52:24.618 INFO:teuthology.orchestra.run.trial066.stdout:kernel.core_pattern = core 2026-02-25T08:52:24.635 DEBUG:teuthology.orchestra.run.trial042:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:52:24.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:52:24.659 DEBUG:teuthology.orchestra.run.trial066:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:52:24.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:52:24.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-25T08:52:24.710 INFO:teuthology.task.internal:Transferring archived files... 2026-02-25T08:52:24.711 DEBUG:teuthology.misc:Transferring archived files from trial042:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053/remote/trial042 2026-02-25T08:52:24.712 DEBUG:teuthology.orchestra.run.trial042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T08:52:24.745 DEBUG:teuthology.misc:Transferring archived files from trial066:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69053/remote/trial066 2026-02-25T08:52:24.745 DEBUG:teuthology.orchestra.run.trial066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T08:52:24.778 INFO:teuthology.task.internal:Removing archive directory... 2026-02-25T08:52:24.778 DEBUG:teuthology.orchestra.run.trial042:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T08:52:24.781 DEBUG:teuthology.orchestra.run.trial066:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T08:52:24.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-25T08:52:24.847 INFO:teuthology.task.internal:Not uploading archives. 2026-02-25T08:52:24.847 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-25T08:52:24.860 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-25T08:52:24.860 DEBUG:teuthology.orchestra.run.trial042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T08:52:24.862 DEBUG:teuthology.orchestra.run.trial066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T08:52:24.877 INFO:teuthology.orchestra.run.trial042.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 08:52 /home/ubuntu/cephtest 2026-02-25T08:52:24.884 INFO:teuthology.orchestra.run.trial066.stdout: 920835 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 08:52 /home/ubuntu/cephtest 2026-02-25T08:52:24.885 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-25T08:52:24.899 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-25T08:52:24.912 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial042 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2026-02-25T08:52:24.915 INFO:teuthology.run:Summary data: description: crimson-rados/thrash/{0-crimson_install 0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} clusters/{crimson-fixed-2} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} thrashers/{default} thrashosds-health workloads/admin_socket_objecter_requests} duration: 2789.8566768169403 failure_reason: 'Command failed on trial042 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json''' flavor: debug owner: scheduled_matan@soko04.front.sepia.ceph.com status: fail success: false 2026-02-25T08:52:24.915 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T08:52:25.022 INFO:teuthology.run:FAIL