2026-02-20T11:23:19.634 INFO:root:teuthology version: 1.2.4.dev11+ge6d9099ce 2026-02-20T11:23:19.648 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T11:23:19.729 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60522 branch: wip-perezjos-crimson-only-19-02-2026-PR64439 description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_api_tests} email: null first_in_suite: false flavor: debug job_id: '60522' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-perezjos-crimson-only-19-02-2026-PR64439 ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_APP_NOT_ENABLED\) - is full \(reached quota - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(POOL_FULL\) sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 445d5c9ba18a4040e70e25303230c36c77773acb owner: scheduled_jjperez@soko04.front.sepia.ceph.com priority: 101 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 1736 sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 445d5c9ba18a4040e70e25303230c36c77773acb targets: trial072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOMpdyf701I1KsmqXu/xjfUFX9btE4RLcSi9jddRmtpfMuLLSpLP1swn2ZraA6jVP6hLSMyFL0lvKVvEaxqE2JE= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - rbd/test_librbd.sh env: RBD_FEATURES: '61' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: e6d9099ce03dc21193fda653159ec82465c3a94a timestamp: 2026-02-20_10:21:31 tube: trial user: jjperez verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T11:23:19.729 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa; will attempt to use it 2026-02-20T11:23:19.729 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks 2026-02-20T11:23:19.729 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T11:23:19.731 INFO:teuthology.task.internal:Checking packages... 2026-02-20T11:23:19.731 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4' 2026-02-20T11:23:19.731 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T11:23:19.731 INFO:teuthology.packaging:ref: None 2026-02-20T11:23:19.731 INFO:teuthology.packaging:tag: None 2026-02-20T11:23:19.731 INFO:teuthology.packaging:branch: wip-perezjos-crimson-only-19-02-2026-PR64439 2026-02-20T11:23:19.731 INFO:teuthology.packaging:sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:23:19.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-perezjos-crimson-only-19-02-2026-PR64439 2026-02-20T11:23:19.847 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5357.g0c3148ca 2026-02-20T11:23:19.848 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T11:23:19.859 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T11:23:19.860 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T11:23:19.873 INFO:teuthology.task.internal:Saving configuration 2026-02-20T11:23:19.886 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T11:23:19.899 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T11:23:19.975 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial072.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60522', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 11:19:53.656219', 'locked_by': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOMpdyf701I1KsmqXu/xjfUFX9btE4RLcSi9jddRmtpfMuLLSpLP1swn2ZraA6jVP6hLSMyFL0lvKVvEaxqE2JE='} 2026-02-20T11:23:19.975 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T11:23:19.986 INFO:teuthology.task.internal:roles: ubuntu@trial072.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T11:23:19.986 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T11:23:20.070 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x73363c0389a0>, signals=[15]) 2026-02-20T11:23:20.070 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T11:23:20.082 INFO:teuthology.task.internal:Opening connections... 2026-02-20T11:23:20.082 DEBUG:teuthology.task.internal:connecting to ubuntu@trial072.front.sepia.ceph.com 2026-02-20T11:23:20.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T11:23:20.150 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T11:23:20.161 DEBUG:teuthology.orchestra.run.trial072:> uname -m 2026-02-20T11:23:20.223 INFO:teuthology.orchestra.run.trial072.stdout:x86_64 2026-02-20T11:23:20.223 DEBUG:teuthology.orchestra.run.trial072:> cat /etc/os-release 2026-02-20T11:23:20.278 INFO:teuthology.orchestra.run.trial072.stdout:NAME="CentOS Stream" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:VERSION="9" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:ID="centos" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:ID_LIKE="rhel fedora" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:VERSION_ID="9" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:PLATFORM_ID="platform:el9" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:ANSI_COLOR="0;31" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:LOGO="fedora-logo-icon" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:HOME_URL="https://centos.org/" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T11:23:20.279 INFO:teuthology.orchestra.run.trial072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T11:23:20.280 INFO:teuthology.lock.ops:Updating trial072.front.sepia.ceph.com on lock server 2026-02-20T11:23:20.360 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T11:23:20.372 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T11:23:20.384 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T11:23:20.384 DEBUG:teuthology.orchestra.run.trial072:> test '!' -e /home/ubuntu/cephtest 2026-02-20T11:23:20.399 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T11:23:20.410 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T11:23:20.410 DEBUG:teuthology.orchestra.run.trial072:> test -z $(ls -A /var/lib/ceph) 2026-02-20T11:23:20.457 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T11:23:20.556 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T11:23:20.575 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-20T11:23:20.575 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-20T11:23:20.576 DEBUG:teuthology.orchestra.run.trial072:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T11:23:20.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:23:20.591 DEBUG:teuthology.orchestra.run.trial072:> uname -r 2026-02-20T11:23:20.647 INFO:teuthology.orchestra.run.trial072.stdout:5.14.0-681.el9.x86_64 2026-02-20T11:23:20.647 INFO:teuthology.task.kernel:Running kernel on trial072: 5.14.0-681.el9.x86_64 2026-02-20T11:23:20.648 DEBUG:teuthology.orchestra.run.trial072:> sudo yum install -y kernel 2026-02-20T11:23:22.322 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.9 MB 00:01 2026-02-20T11:23:25.459 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-20T11:23:29.093 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 7.8 MB 00:01 2026-02-20T11:23:30.382 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 20 kB 00:00 2026-02-20T11:23:36.726 INFO:teuthology.orchestra.run.trial072.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 20 MB 00:06 2026-02-20T11:23:39.912 INFO:teuthology.orchestra.run.trial072.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-20T11:23:40.918 INFO:teuthology.orchestra.run.trial072.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T11:23:40.918 INFO:teuthology.orchestra.run.trial072.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T11:23:40.937 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:23:40.938 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:23:40.938 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:23:40.962 DEBUG:teuthology.orchestra.run.trial072:> echo no | sudo yum reinstall kernel || true 2026-02-20T11:23:41.265 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:23:39 AM UTC. 2026-02-20T11:23:41.346 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:23:41.348 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:23:41.348 INFO:teuthology.orchestra.run.trial072.stdout: Package Architecture Version Repository Size 2026-02-20T11:23:41.348 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:23:41.348 INFO:teuthology.orchestra.run.trial072.stdout:Reinstalling: 2026-02-20T11:23:41.349 INFO:teuthology.orchestra.run.trial072.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T11:23:41.349 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:23:41.349 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:23:41.349 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:23:41.349 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:23:41.349 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 803 k 2026-02-20T11:23:41.349 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 0 2026-02-20T11:23:41.350 INFO:teuthology.orchestra.run.trial072.stderr:Operation aborted. 2026-02-20T11:23:41.367 INFO:teuthology.orchestra.run.trial072.stdout:Is this ok [y/N]: 2026-02-20T11:23:41.367 DEBUG:teuthology.orchestra.run.trial072:> sudo yum reinstall -y kernel || true 2026-02-20T11:23:41.672 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:23:39 AM UTC. 2026-02-20T11:23:41.752 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout: Package Architecture Version Repository Size 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout:Reinstalling: 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:23:41.753 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:23:41.754 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 803 k 2026-02-20T11:23:41.754 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 0 2026-02-20T11:23:41.754 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:23:41.971 INFO:teuthology.orchestra.run.trial072.stdout:kernel-5.14.0-681.el9.x86_64.rpm 6.1 MB/s | 803 kB 00:00 2026-02-20T11:23:41.971 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:23:41.971 INFO:teuthology.orchestra.run.trial072.stdout:Total 3.6 MB/s | 803 kB 00:00 2026-02-20T11:23:41.981 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:23:42.014 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:23:42.014 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:23:42.019 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:23:42.019 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:23:42.043 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:23:42.047 INFO:teuthology.orchestra.run.trial072.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T11:23:42.047 INFO:teuthology.orchestra.run.trial072.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:23:42.162 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:23:42.162 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T11:23:42.215 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:23:42.215 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:23:42.215 INFO:teuthology.orchestra.run.trial072.stdout:Reinstalled: 2026-02-20T11:23:42.215 INFO:teuthology.orchestra.run.trial072.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T11:23:42.216 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:23:42.216 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:23:42.243 DEBUG:teuthology.orchestra.run.trial072:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T11:23:42.267 INFO:teuthology.orchestra.run.trial072.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T11:23:42.268 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T11:23:42.268 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T11:23:42.268 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T11:23:42.268 DEBUG:teuthology.orchestra.run.trial072:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T11:23:42.333 INFO:teuthology.orchestra.run.trial072.stdout:ttyS1 2026-02-20T11:23:42.360 DEBUG:teuthology.parallel:result is None 2026-02-20T11:23:42.360 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T11:23:42.373 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T11:23:42.373 DEBUG:teuthology.orchestra.run.trial072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T11:23:42.416 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T11:23:42.427 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T11:23:42.441 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T11:23:42.441 DEBUG:teuthology.orchestra.run.trial072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T11:23:42.478 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T11:23:42.489 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T11:23:42.490 DEBUG:teuthology.orchestra.run.trial072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T11:23:42.537 INFO:teuthology.orchestra.run.trial072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T11:23:42.546 INFO:teuthology.orchestra.run.trial072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T11:23:42.548 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T11:23:42.561 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T11:23:42.561 DEBUG:teuthology.orchestra.run.trial072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T11:23:42.613 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T11:23:42.630 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T11:23:42.630 DEBUG:teuthology.orchestra.run.trial072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T11:23:42.670 DEBUG:teuthology.orchestra.run.trial072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:23:42.829 DEBUG:teuthology.orchestra.run.trial072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:23:42.854 DEBUG:teuthology.orchestra.run.trial072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:23:42.995 DEBUG:teuthology.orchestra.run.trial072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:23:43.019 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:23:43.019 DEBUG:teuthology.orchestra.run.trial072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T11:23:43.082 DEBUG:teuthology.orchestra.run.trial072:> sudo service rsyslog restart 2026-02-20T11:23:43.149 INFO:teuthology.orchestra.run.trial072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T11:23:43.541 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T11:23:43.553 INFO:teuthology.task.internal:Starting timer... 2026-02-20T11:23:43.554 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T11:23:43.574 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T11:23:43.591 DEBUG:teuthology.orchestra.run.trial072:> sudo service auditd rotate 2026-02-20T11:23:43.619 INFO:teuthology.orchestra.run.trial072.stdout:Rotating logs: 2026-02-20T11:23:43.621 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T11:23:43.621 DEBUG:teuthology.orchestra.run.trial072:> /usr/sbin/getenforce 2026-02-20T11:23:43.677 INFO:teuthology.orchestra.run.trial072.stdout:Permissive 2026-02-20T11:23:43.678 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial072.front.sepia.ceph.com': 'permissive'} 2026-02-20T11:23:43.678 DEBUG:teuthology.orchestra.run.trial072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T11:23:43.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:23:43.744 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T11:23:43.744 DEBUG:teuthology.orchestra.run.trial072:> sudo /usr/sbin/setenforce permissive 2026-02-20T11:23:43.807 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T11:23:43.820 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T11:23:43.831 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T11:23:43.833 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial072.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T11:26:25.847 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial072.front.sepia.ceph.com')] 2026-02-20T11:26:25.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial072.front.sepia.ceph.com' 2026-02-20T11:26:25.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T11:26:25.960 DEBUG:teuthology.orchestra.run.trial072:> true 2026-02-20T11:26:26.040 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial072.front.sepia.ceph.com' 2026-02-20T11:26:26.040 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T11:26:26.052 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T11:26:26.052 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T11:26:26.052 DEBUG:teuthology.orchestra.run.trial072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T11:26:26.108 INFO:teuthology.orchestra.run.trial072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T11:26:26.120 INFO:teuthology.orchestra.run.trial072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T11:26:26.141 INFO:teuthology.orchestra.run.trial072.stderr:sudo: ntpd: command not found 2026-02-20T11:26:26.150 INFO:teuthology.orchestra.run.trial072.stdout:506 Cannot talk to daemon 2026-02-20T11:26:26.162 INFO:teuthology.orchestra.run.trial072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T11:26:26.174 INFO:teuthology.orchestra.run.trial072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T11:26:26.216 INFO:teuthology.orchestra.run.trial072.stderr:bash: line 1: ntpq: command not found 2026-02-20T11:26:26.218 INFO:teuthology.orchestra.run.trial072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T11:26:26.218 INFO:teuthology.orchestra.run.trial072.stdout:=============================================================================== 2026-02-20T11:26:26.219 INFO:teuthology.run_tasks:Running task install... 2026-02-20T11:26:26.235 DEBUG:teuthology.task.install:project ceph 2026-02-20T11:26:26.235 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}} 2026-02-20T11:26:26.235 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'} 2026-02-20T11:26:26.235 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T11:26:26.244 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T11:26:26.244 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-20T11:26:26.244 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'tag': None, 'wait_for_package': False} 2026-02-20T11:26:26.245 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:26:26.406 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/ 2026-02-20T11:26:26.406 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5357.g0c3148ca 2026-02-20T11:26:26.781 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T11:26:26.782 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:26:26.782 DEBUG:teuthology.orchestra.run.trial072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T11:26:26.808 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-20T11:26:26.808 DEBUG:teuthology.orchestra.run.trial072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T11:26:26.874 DEBUG:teuthology.orchestra.run.trial072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T11:26:26.947 DEBUG:teuthology.orchestra.run.trial072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T11:26:27.011 INFO:teuthology.orchestra.run.trial072.stdout:check_obsoletes = 1 2026-02-20T11:26:27.012 DEBUG:teuthology.orchestra.run.trial072:> sudo yum clean all 2026-02-20T11:26:27.172 INFO:teuthology.orchestra.run.trial072.stdout:41 files removed 2026-02-20T11:26:27.187 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-radosgw 2026-02-20T11:26:27.946 INFO:teuthology.orchestra.run.trial072.stdout:ceph packages for x86_64 153 kB/s | 94 kB 00:00 2026-02-20T11:26:28.506 INFO:teuthology.orchestra.run.trial072.stdout:ceph noarch packages 46 kB/s | 24 kB 00:00 2026-02-20T11:26:29.037 INFO:teuthology.orchestra.run.trial072.stdout:ceph source packages 4.1 kB/s | 2.1 kB 00:00 2026-02-20T11:26:29.977 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.9 MB 00:00 2026-02-20T11:26:33.694 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 27 MB 00:03 2026-02-20T11:26:37.361 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 7.8 MB 00:01 2026-02-20T11:26:38.367 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 20 kB 00:00 2026-02-20T11:26:38.904 INFO:teuthology.orchestra.run.trial072.stdout:Extra Packages for Enterprise Linux 43 MB/s | 20 MB 00:00 2026-02-20T11:26:42.139 INFO:teuthology.orchestra.run.trial072.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-20T11:26:43.346 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:26:43.347 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:26:43.347 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repository Size 2026-02-20T11:26:43.347 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout: ceph-radosgw x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 11 M 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout:Upgrading: 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout: librados2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 3.3 M 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout: librbd1 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.7 M 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout:Installing dependencies: 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout: ceph-base x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 5.1 M 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout: ceph-common x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 21 M 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout: ceph-selinux x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 25 k 2026-02-20T11:26:43.348 INFO:teuthology.orchestra.run.trial072.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: libcephfs2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 860 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: libradosstriper1 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 263 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: librgw2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 5.6 M 2026-02-20T11:26:43.349 INFO:teuthology.orchestra.run.trial072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: python3-ceph-argparse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 45 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: python3-ceph-common x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 185 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: python3-cephfs x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 204 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: python3-rados x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 324 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: python3-rbd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 305 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: python3-rgw x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 100 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T11:26:43.350 INFO:teuthology.orchestra.run.trial072.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout:Installing weak dependencies: 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout: libcephfs-daemon x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 37 k 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 26 k 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout:Install 35 Packages 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout:Upgrade 2 Packages 2026-02-20T11:26:43.351 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:26:43.352 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 61 M 2026-02-20T11:26:43.352 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:26:45.771 INFO:teuthology.orchestra.run.trial072.stdout:(1/37): ceph-radosgw-20.3.0-5357.g0c3148ca.el9. 5.0 MB/s | 11 MB 00:02 2026-02-20T11:26:45.828 INFO:teuthology.orchestra.run.trial072.stdout:(2/37): ceph-selinux-20.3.0-5357.g0c3148ca.el9. 439 kB/s | 25 kB 00:00 2026-02-20T11:26:45.884 INFO:teuthology.orchestra.run.trial072.stdout:(3/37): libcephfs-daemon-20.3.0-5357.g0c3148ca. 666 kB/s | 37 kB 00:00 2026-02-20T11:26:45.940 INFO:teuthology.orchestra.run.trial072.stdout:(4/37): libcephfs-proxy2-20.3.0-5357.g0c3148ca. 456 kB/s | 26 kB 00:00 2026-02-20T11:26:46.073 INFO:teuthology.orchestra.run.trial072.stdout:(5/37): libcephfs2-20.3.0-5357.g0c3148ca.el9.x8 6.3 MB/s | 860 kB 00:00 2026-02-20T11:26:46.142 INFO:teuthology.orchestra.run.trial072.stdout:(6/37): libradosstriper1-20.3.0-5357.g0c3148ca. 3.7 MB/s | 263 kB 00:00 2026-02-20T11:26:47.224 INFO:teuthology.orchestra.run.trial072.stdout:(7/37): librgw2-20.3.0-5357.g0c3148ca.el9.x86_6 5.2 MB/s | 5.6 MB 00:01 2026-02-20T11:26:47.279 INFO:teuthology.orchestra.run.trial072.stdout:(8/37): python3-ceph-argparse-20.3.0-5357.g0c31 820 kB/s | 45 kB 00:00 2026-02-20T11:26:47.336 INFO:teuthology.orchestra.run.trial072.stdout:(9/37): python3-ceph-common-20.3.0-5357.g0c3148 3.1 MB/s | 185 kB 00:00 2026-02-20T11:26:47.393 INFO:teuthology.orchestra.run.trial072.stdout:(10/37): python3-cephfs-20.3.0-5357.g0c3148ca.e 3.5 MB/s | 204 kB 00:00 2026-02-20T11:26:47.453 INFO:teuthology.orchestra.run.trial072.stdout:(11/37): python3-rados-20.3.0-5357.g0c3148ca.el 5.3 MB/s | 324 kB 00:00 2026-02-20T11:26:47.513 INFO:teuthology.orchestra.run.trial072.stdout:(12/37): python3-rbd-20.3.0-5357.g0c3148ca.el9. 5.0 MB/s | 305 kB 00:00 2026-02-20T11:26:47.568 INFO:teuthology.orchestra.run.trial072.stdout:(13/37): python3-rgw-20.3.0-5357.g0c3148ca.el9. 1.8 MB/s | 100 kB 00:00 2026-02-20T11:26:48.126 INFO:teuthology.orchestra.run.trial072.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 59 kB/s | 33 kB 00:00 2026-02-20T11:26:48.579 INFO:teuthology.orchestra.run.trial072.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 2.5 MB/s | 1.2 MB 00:00 2026-02-20T11:26:48.686 INFO:teuthology.orchestra.run.trial072.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 874 kB/s | 93 kB 00:00 2026-02-20T11:26:49.037 INFO:teuthology.orchestra.run.trial072.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 296 kB/s | 104 kB 00:00 2026-02-20T11:26:49.317 INFO:teuthology.orchestra.run.trial072.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 585 kB/s | 164 kB 00:00 2026-02-20T11:26:49.595 INFO:teuthology.orchestra.run.trial072.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 577 kB/s | 160 kB 00:00 2026-02-20T11:26:49.824 INFO:teuthology.orchestra.run.trial072.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 198 kB/s | 45 kB 00:00 2026-02-20T11:26:50.235 INFO:teuthology.orchestra.run.trial072.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2026-02-20T11:26:50.451 INFO:teuthology.orchestra.run.trial072.stdout:(22/37): ceph-base-20.3.0-5357.g0c3148ca.el9.x8 757 kB/s | 5.1 MB 00:06 2026-02-20T11:26:50.565 INFO:teuthology.orchestra.run.trial072.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 887 kB/s | 292 kB 00:00 2026-02-20T11:26:50.770 INFO:teuthology.orchestra.run.trial072.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 749 kB/s | 240 kB 00:00 2026-02-20T11:26:50.795 INFO:teuthology.orchestra.run.trial072.stdout:(25/37): qatlib-service-25.08.0-2.el9.x86_64.rp 161 kB/s | 37 kB 00:00 2026-02-20T11:26:50.966 INFO:teuthology.orchestra.run.trial072.stdout:(26/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.8 MB/s | 308 kB 00:00 2026-02-20T11:26:51.002 INFO:teuthology.orchestra.run.trial072.stdout:(27/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 287 kB/s | 66 kB 00:00 2026-02-20T11:26:51.045 INFO:teuthology.orchestra.run.trial072.stdout:(28/37): libarrow-9.0.0-15.el9.x86_64.rpm 56 MB/s | 4.4 MB 00:00 2026-02-20T11:26:51.045 INFO:teuthology.orchestra.run.trial072.stdout:(29/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 575 kB/s | 25 kB 00:00 2026-02-20T11:26:51.060 INFO:teuthology.orchestra.run.trial072.stdout:(30/37): liboath-2.6.12-1.el9.x86_64.rpm 3.2 MB/s | 49 kB 00:00 2026-02-20T11:26:51.078 INFO:teuthology.orchestra.run.trial072.stdout:(31/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 46 MB/s | 838 kB 00:00 2026-02-20T11:26:51.079 INFO:teuthology.orchestra.run.trial072.stdout:(32/37): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2026-02-20T11:26:51.094 INFO:teuthology.orchestra.run.trial072.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 12 MB/s | 191 kB 00:00 2026-02-20T11:26:51.157 INFO:teuthology.orchestra.run.trial072.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 20 MB/s | 1.6 MB 00:00 2026-02-20T11:26:51.362 INFO:teuthology.orchestra.run.trial072.stdout:(35/37): ceph-common-20.3.0-5357.g0c3148ca.el9. 2.7 MB/s | 21 MB 00:07 2026-02-20T11:26:52.187 INFO:teuthology.orchestra.run.trial072.stdout:(36/37): librbd1-20.3.0-5357.g0c3148ca.el9.x86_ 2.6 MB/s | 2.7 MB 00:01 2026-02-20T11:26:53.795 INFO:teuthology.orchestra.run.trial072.stdout:(37/37): librados2-20.3.0-5357.g0c3148ca.el9.x8 1.2 MB/s | 3.3 MB 00:02 2026-02-20T11:26:53.798 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:26:53.798 INFO:teuthology.orchestra.run.trial072.stdout:Total 5.9 MB/s | 61 MB 00:10 2026-02-20T11:26:53.900 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:26:53.919 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:26:53.919 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:26:54.170 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:26:54.170 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:26:54.468 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:26:54.608 INFO:teuthology.orchestra.run.trial072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-20T11:26:54.612 INFO:teuthology.orchestra.run.trial072.stdout: Upgrading : librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:26:54.664 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:26:54.665 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:26:54.678 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:26:54.680 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_6 4/39 2026-02-20T11:26:54.681 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:26:54.709 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:26:54.713 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 6/39 2026-02-20T11:26:54.724 INFO:teuthology.orchestra.run.trial072.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-20T11:26:54.727 INFO:teuthology.orchestra.run.trial072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-20T11:26:54.730 INFO:teuthology.orchestra.run.trial072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-20T11:26:54.734 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-20T11:26:54.745 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9. 11/39 2026-02-20T11:26:54.758 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/39 2026-02-20T11:26:54.759 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 13/39 2026-02-20T11:26:54.784 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 13/39 2026-02-20T11:26:54.789 INFO:teuthology.orchestra.run.trial072.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-20T11:26:54.820 INFO:teuthology.orchestra.run.trial072.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-20T11:26:54.824 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-20T11:26:54.830 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-20T11:26:54.832 INFO:teuthology.orchestra.run.trial072.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-20T11:26:54.837 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-20T11:26:54.951 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T11:26:54.952 INFO:teuthology.orchestra.run.trial072.stdout: Upgrading : librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 21/39 2026-02-20T11:26:54.979 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 21/39 2026-02-20T11:26:54.982 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 22/39 2026-02-20T11:26:54.988 INFO:teuthology.orchestra.run.trial072.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-20T11:26:55.238 INFO:teuthology.orchestra.run.trial072.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-20T11:26:55.239 INFO:teuthology.orchestra.run.trial072.stdout: Installing : librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 25/39 2026-02-20T11:26:55.256 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 25/39 2026-02-20T11:26:55.261 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 26/39 2026-02-20T11:26:55.265 INFO:teuthology.orchestra.run.trial072.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-20T11:26:55.268 INFO:teuthology.orchestra.run.trial072.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-20T11:26:55.269 INFO:teuthology.orchestra.run.trial072.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T11:26:55.286 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T11:26:55.401 INFO:teuthology.orchestra.run.trial072.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-20T11:26:55.403 INFO:teuthology.orchestra.run.trial072.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T11:26:55.686 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T11:26:55.687 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-20T11:26:55.687 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:26:55.730 INFO:teuthology.orchestra.run.trial072.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-20T11:26:55.732 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x8 33/39 2026-02-20T11:26:56.676 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:26:56.679 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:26:56.940 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:26:56.943 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 35/39 2026-02-20T11:26:56.984 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 35/39 2026-02-20T11:26:56.984 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T11:26:56.984 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T11:26:56.984 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:26:56.988 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:27:01.538 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:27:01.538 INFO:teuthology.orchestra.run.trial072.stdout:skipping the directory /sys 2026-02-20T11:27:01.538 INFO:teuthology.orchestra.run.trial072.stdout:skipping the directory /proc 2026-02-20T11:27:01.538 INFO:teuthology.orchestra.run.trial072.stdout:skipping the directory /mnt 2026-02-20T11:27:01.538 INFO:teuthology.orchestra.run.trial072.stdout:skipping the directory /var/tmp 2026-02-20T11:27:01.538 INFO:teuthology.orchestra.run.trial072.stdout:skipping the directory /home 2026-02-20T11:27:01.538 INFO:teuthology.orchestra.run.trial072.stdout:skipping the directory /root 2026-02-20T11:27:01.539 INFO:teuthology.orchestra.run.trial072.stdout:skipping the directory /tmp 2026-02-20T11:27:01.539 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:27:02.031 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 37/39 2026-02-20T11:27:02.055 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 37/39 2026-02-20T11:27:02.055 INFO:teuthology.orchestra.run.trial072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:27:02.055 INFO:teuthology.orchestra.run.trial072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T11:27:02.055 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T11:27:02.055 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T11:27:02.056 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:27:02.057 INFO:teuthology.orchestra.run.trial072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T11:27:02.070 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T11:27:02.070 INFO:teuthology.orchestra.run.trial072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:27:02.127 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:27:02.128 INFO:teuthology.orchestra.run.trial072.stdout:Creating group 'qat' with GID 992. 2026-02-20T11:27:02.128 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:27:02.441 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/39 2026-02-20T11:27:02.441 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:27:02.442 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:27:02.442 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/39 2026-02-20T11:27:02.442 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:27:02.442 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 6/39 2026-02-20T11:27:02.442 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 7/39 2026-02-20T11:27:02.442 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 8/39 2026-02-20T11:27:02.442 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 9/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9. 10/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x8 11/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 13/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 14/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 15/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-20T11:27:02.443 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-20T11:27:02.444 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-20T11:27:02.445 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 38/39 2026-02-20T11:27:02.503 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:27:02.503 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:27:02.503 INFO:teuthology.orchestra.run.trial072.stdout:Upgraded: 2026-02-20T11:27:02.503 INFO:teuthology.orchestra.run.trial072.stdout: librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.503 INFO:teuthology.orchestra.run.trial072.stdout: librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.503 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:27:02.503 INFO:teuthology.orchestra.run.trial072.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T11:27:02.503 INFO:teuthology.orchestra.run.trial072.stdout: ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T11:27:02.504 INFO:teuthology.orchestra.run.trial072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.505 INFO:teuthology.orchestra.run.trial072.stdout: python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:27:02.506 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:27:02.545 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-test 2026-02-20T11:27:02.859 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:20 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:27:02.942 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repository Size 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout: ceph-test x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 127 M 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout:Installing dependencies: 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout: libcephsqlite x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 153 k 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-20T11:27:02.943 INFO:teuthology.orchestra.run.trial072.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout:Install 12 Packages 2026-02-20T11:27:02.944 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:27:02.945 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 132 M 2026-02-20T11:27:02.945 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 989 M 2026-02-20T11:27:02.945 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:27:03.599 INFO:teuthology.orchestra.run.trial072.stdout:(1/12): libcephsqlite-20.3.0-5357.g0c3148ca.el9 341 kB/s | 153 kB 00:00 2026-02-20T11:27:04.128 INFO:teuthology.orchestra.run.trial072.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 43 kB/s | 23 kB 00:00 2026-02-20T11:27:04.278 INFO:teuthology.orchestra.run.trial072.stdout:(3/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 1.9 MB/s | 2.1 MB 00:01 2026-02-20T11:27:04.455 INFO:teuthology.orchestra.run.trial072.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 2.9 MB/s | 525 kB 00:00 2026-02-20T11:27:04.476 INFO:teuthology.orchestra.run.trial072.stdout:(5/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 270 kB/s | 94 kB 00:00 2026-02-20T11:27:04.630 INFO:teuthology.orchestra.run.trial072.stdout:(6/12): libubsan-11.5.0-14.el9.x86_64.rpm 1.0 MB/s | 176 kB 00:00 2026-02-20T11:27:04.737 INFO:teuthology.orchestra.run.trial072.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 895 kB/s | 233 kB 00:00 2026-02-20T11:27:04.913 INFO:teuthology.orchestra.run.trial072.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 1.7 MB/s | 303 kB 00:00 2026-02-20T11:27:05.087 INFO:teuthology.orchestra.run.trial072.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 366 kB/s | 64 kB 00:00 2026-02-20T11:27:05.413 INFO:teuthology.orchestra.run.trial072.stdout:(10/12): protobuf-3.14.0-17.el9.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2026-02-20T11:27:05.448 INFO:teuthology.orchestra.run.trial072.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 336 kB/s | 122 kB 00:00 2026-02-20T11:28:11.569 INFO:teuthology.orchestra.run.trial072.stdout:(12/12): ceph-test-20.3.0-5357.g0c3148ca.el9.x8 1.9 MB/s | 127 MB 01:08 2026-02-20T11:28:11.572 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:28:11.572 INFO:teuthology.orchestra.run.trial072.stdout:Total 1.9 MB/s | 132 MB 01:08 2026-02-20T11:28:11.641 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:28:11.653 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:28:11.653 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:28:12.086 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:28:12.086 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:28:12.549 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:28:12.557 INFO:teuthology.orchestra.run.trial072.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-20T11:28:12.566 INFO:teuthology.orchestra.run.trial072.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-20T11:28:12.571 INFO:teuthology.orchestra.run.trial072.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-20T11:28:12.575 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-20T11:28:12.578 INFO:teuthology.orchestra.run.trial072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-20T11:28:12.582 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-20T11:28:12.583 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T11:28:12.600 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T11:28:12.602 INFO:teuthology.orchestra.run.trial072.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-20T11:28:12.613 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-20T11:28:12.621 INFO:teuthology.orchestra.run.trial072.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-20T11:28:12.622 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 11/12 2026-02-20T11:28:19.588 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 11/12 2026-02-20T11:28:19.606 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/12 2026-02-20T11:28:20.476 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/12 2026-02-20T11:28:20.476 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/12 2026-02-20T11:28:20.476 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/12 2026-02-20T11:28:20.476 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-20T11:28:20.476 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-20T11:28:20.477 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-20T11:28:20.477 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-20T11:28:20.477 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-20T11:28:20.477 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-20T11:28:20.477 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-20T11:28:20.477 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-20T11:28:20.477 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-20T11:28:20.530 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout: ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout: libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-20T11:28:20.531 INFO:teuthology.orchestra.run.trial072.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T11:28:20.532 INFO:teuthology.orchestra.run.trial072.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-20T11:28:20.532 INFO:teuthology.orchestra.run.trial072.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T11:28:20.532 INFO:teuthology.orchestra.run.trial072.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T11:28:20.532 INFO:teuthology.orchestra.run.trial072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T11:28:20.532 INFO:teuthology.orchestra.run.trial072.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-20T11:28:20.532 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:28:20.532 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:28:20.572 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph 2026-02-20T11:28:20.884 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:01:38 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:28:20.973 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:28:20.975 INFO:teuthology.orchestra.run.trial072.stdout:======================================================================================= 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repository Size 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout:======================================================================================= 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout: ceph x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 6.5 k 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout:Installing dependencies: 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 57 k 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mds x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.0 M 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 937 k 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 297 k 2026-02-20T11:28:20.976 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mon x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 4.9 M 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: ceph-osd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 163 k 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: ceph-osd-classic x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 15 M 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: ceph-osd-crimson x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 88 M 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 17 k 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: cephadm noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 1.0 M 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T11:28:20.977 INFO:teuthology.orchestra.run.trial072.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T11:28:20.978 INFO:teuthology.orchestra.run.trial072.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T11:28:20.979 INFO:teuthology.orchestra.run.trial072.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T11:28:20.980 INFO:teuthology.orchestra.run.trial072.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T11:28:20.981 INFO:teuthology.orchestra.run.trial072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T11:28:20.982 INFO:teuthology.orchestra.run.trial072.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout:Installing weak dependencies: 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 184 k 2026-02-20T11:28:20.983 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 51 M 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 7.4 M 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 22 k 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-rook noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 50 k 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout: ceph-volume noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 301 k 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout:======================================================================================= 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout:Install 96 Packages 2026-02-20T11:28:20.984 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:28:20.985 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 226 M 2026-02-20T11:28:20.985 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 1.5 G 2026-02-20T11:28:20.985 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:28:21.578 INFO:teuthology.orchestra.run.trial072.stdout:(1/96): ceph-20.3.0-5357.g0c3148ca.el9.x86_64.r 23 kB/s | 6.5 kB 00:00 2026-02-20T11:28:22.253 INFO:teuthology.orchestra.run.trial072.stdout:(2/96): ceph-mgr-20.3.0-5357.g0c3148ca.el9.x86_ 981 kB/s | 937 kB 00:00 2026-02-20T11:28:22.283 INFO:teuthology.orchestra.run.trial072.stdout:(3/96): ceph-mds-20.3.0-5357.g0c3148ca.el9.x86_ 2.1 MB/s | 2.0 MB 00:00 2026-02-20T11:28:22.384 INFO:teuthology.orchestra.run.trial072.stdout:(4/96): ceph-osd-20.3.0-5357.g0c3148ca.el9.x86_ 1.2 MB/s | 163 kB 00:00 2026-02-20T11:28:23.259 INFO:teuthology.orchestra.run.trial072.stdout:(5/96): ceph-mon-20.3.0-5357.g0c3148ca.el9.x86_ 2.9 MB/s | 4.9 MB 00:01 2026-02-20T11:28:23.323 INFO:teuthology.orchestra.run.trial072.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5357.g0c 895 kB/s | 57 kB 00:00 2026-02-20T11:28:23.454 INFO:teuthology.orchestra.run.trial072.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5357.g0c3148ca. 1.4 MB/s | 184 kB 00:00 2026-02-20T11:28:25.734 INFO:teuthology.orchestra.run.trial072.stdout:(8/96): ceph-osd-classic-20.3.0-5357.g0c3148ca. 4.3 MB/s | 15 MB 00:03 2026-02-20T11:28:27.932 INFO:teuthology.orchestra.run.trial072.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 3.4 MB/s | 7.4 MB 00:02 2026-02-20T11:28:28.000 INFO:teuthology.orchestra.run.trial072.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5357.g0c3148 332 kB/s | 22 kB 00:00 2026-02-20T11:28:28.136 INFO:teuthology.orchestra.run.trial072.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5357.g0c3 2.1 MB/s | 297 kB 00:00 2026-02-20T11:28:28.203 INFO:teuthology.orchestra.run.trial072.stdout:(12/96): ceph-mgr-rook-20.3.0-5357.g0c3148ca.el 755 kB/s | 50 kB 00:00 2026-02-20T11:28:28.269 INFO:teuthology.orchestra.run.trial072.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5357.g0c 261 kB/s | 17 kB 00:00 2026-02-20T11:28:28.408 INFO:teuthology.orchestra.run.trial072.stdout:(14/96): ceph-volume-20.3.0-5357.g0c3148ca.el9. 2.1 MB/s | 301 kB 00:00 2026-02-20T11:28:28.819 INFO:teuthology.orchestra.run.trial072.stdout:(15/96): cephadm-20.3.0-5357.g0c3148ca.el9.noar 2.5 MB/s | 1.0 MB 00:00 2026-02-20T11:28:28.986 INFO:teuthology.orchestra.run.trial072.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.1 MB/s | 351 kB 00:00 2026-02-20T11:28:29.012 INFO:teuthology.orchestra.run.trial072.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.5 MB/s | 40 kB 00:00 2026-02-20T11:28:29.040 INFO:teuthology.orchestra.run.trial072.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 2.6 MB/s | 72 kB 00:00 2026-02-20T11:28:29.103 INFO:teuthology.orchestra.run.trial072.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 12 MB/s | 794 kB 00:00 2026-02-20T11:28:29.132 INFO:teuthology.orchestra.run.trial072.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 6.4 MB/s | 184 kB 00:00 2026-02-20T11:28:29.161 INFO:teuthology.orchestra.run.trial072.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 8.6 MB/s | 253 kB 00:00 2026-02-20T11:28:29.211 INFO:teuthology.orchestra.run.trial072.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 25 MB/s | 1.2 MB 00:00 2026-02-20T11:28:29.238 INFO:teuthology.orchestra.run.trial072.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 3.9 MB/s | 106 kB 00:00 2026-02-20T11:28:29.265 INFO:teuthology.orchestra.run.trial072.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 4.8 MB/s | 135 kB 00:00 2026-02-20T11:28:29.293 INFO:teuthology.orchestra.run.trial072.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 5.4 MB/s | 150 kB 00:00 2026-02-20T11:28:29.320 INFO:teuthology.orchestra.run.trial072.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 4.5 MB/s | 126 kB 00:00 2026-02-20T11:28:29.348 INFO:teuthology.orchestra.run.trial072.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 7.7 MB/s | 218 kB 00:00 2026-02-20T11:28:29.376 INFO:teuthology.orchestra.run.trial072.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 6.4 MB/s | 182 kB 00:00 2026-02-20T11:28:29.427 INFO:teuthology.orchestra.run.trial072.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 5.1 MB/s | 266 kB 00:00 2026-02-20T11:28:29.452 INFO:teuthology.orchestra.run.trial072.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 1.1 MB/s | 30 kB 00:00 2026-02-20T11:28:29.509 INFO:teuthology.orchestra.run.trial072.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 53 MB/s | 3.0 MB 00:00 2026-02-20T11:28:29.518 INFO:teuthology.orchestra.run.trial072.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 1.5 MB/s | 15 kB 00:00 2026-02-20T11:28:29.528 INFO:teuthology.orchestra.run.trial072.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 24 MB/s | 246 kB 00:00 2026-02-20T11:28:29.539 INFO:teuthology.orchestra.run.trial072.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 18 MB/s | 188 kB 00:00 2026-02-20T11:28:29.547 INFO:teuthology.orchestra.run.trial072.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 4.9 MB/s | 42 kB 00:00 2026-02-20T11:28:29.574 INFO:teuthology.orchestra.run.trial072.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 200 MB/s | 5.3 MB 00:00 2026-02-20T11:28:29.606 INFO:teuthology.orchestra.run.trial072.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 183 MB/s | 6.0 MB 00:00 2026-02-20T11:28:29.618 INFO:teuthology.orchestra.run.trial072.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 20 MB/s | 244 kB 00:00 2026-02-20T11:28:29.629 INFO:teuthology.orchestra.run.trial072.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 23 MB/s | 249 kB 00:00 2026-02-20T11:28:29.638 INFO:teuthology.orchestra.run.trial072.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 5.3 MB/s | 48 kB 00:00 2026-02-20T11:28:29.648 INFO:teuthology.orchestra.run.trial072.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 17 MB/s | 177 kB 00:00 2026-02-20T11:28:29.662 INFO:teuthology.orchestra.run.trial072.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 90 MB/s | 1.2 MB 00:00 2026-02-20T11:28:29.671 INFO:teuthology.orchestra.run.trial072.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 3.8 MB/s | 35 kB 00:00 2026-02-20T11:28:29.703 INFO:teuthology.orchestra.run.trial072.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 188 MB/s | 6.1 MB 00:00 2026-02-20T11:28:29.716 INFO:teuthology.orchestra.run.trial072.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 35 MB/s | 442 kB 00:00 2026-02-20T11:28:29.725 INFO:teuthology.orchestra.run.trial072.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 8.0 MB/s | 77 kB 00:00 2026-02-20T11:28:29.736 INFO:teuthology.orchestra.run.trial072.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 25 MB/s | 267 kB 00:00 2026-02-20T11:28:29.747 INFO:teuthology.orchestra.run.trial072.stdout:(48/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 15 MB/s | 159 kB 00:00 2026-02-20T11:28:29.758 INFO:teuthology.orchestra.run.trial072.stdout:(49/96): python3-pyasn1-modules-0.4.8-6.el9.noa 25 MB/s | 279 kB 00:00 2026-02-20T11:28:29.767 INFO:teuthology.orchestra.run.trial072.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 5.7 MB/s | 54 kB 00:00 2026-02-20T11:28:29.865 INFO:teuthology.orchestra.run.trial072.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 197 MB/s | 19 MB 00:00 2026-02-20T11:28:29.872 INFO:teuthology.orchestra.run.trial072.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 5.4 MB/s | 42 kB 00:00 2026-02-20T11:28:29.881 INFO:teuthology.orchestra.run.trial072.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 23 MB/s | 189 kB 00:00 2026-02-20T11:28:29.888 INFO:teuthology.orchestra.run.trial072.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 11 MB/s | 90 kB 00:00 2026-02-20T11:28:29.899 INFO:teuthology.orchestra.run.trial072.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 22 kB 00:00 2026-02-20T11:28:29.919 INFO:teuthology.orchestra.run.trial072.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 43 MB/s | 862 kB 00:00 2026-02-20T11:28:30.109 INFO:teuthology.orchestra.run.trial072.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.8 MB/s | 551 kB 00:00 2026-02-20T11:28:30.124 INFO:teuthology.orchestra.run.trial072.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 1.3 MB/s | 19 kB 00:00 2026-02-20T11:28:30.140 INFO:teuthology.orchestra.run.trial072.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 9.2 MB/s | 151 kB 00:00 2026-02-20T11:28:30.164 INFO:teuthology.orchestra.run.trial072.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 22 MB/s | 548 kB 00:00 2026-02-20T11:28:30.179 INFO:teuthology.orchestra.run.trial072.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 1.9 MB/s | 29 kB 00:00 2026-02-20T11:28:30.195 INFO:teuthology.orchestra.run.trial072.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 3.8 MB/s | 60 kB 00:00 2026-02-20T11:28:30.210 INFO:teuthology.orchestra.run.trial072.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.8 MB/s | 43 kB 00:00 2026-02-20T11:28:30.225 INFO:teuthology.orchestra.run.trial072.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 2.1 MB/s | 32 kB 00:00 2026-02-20T11:28:30.240 INFO:teuthology.orchestra.run.trial072.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 946 kB/s | 14 kB 00:00 2026-02-20T11:28:30.258 INFO:teuthology.orchestra.run.trial072.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 9.9 MB/s | 173 kB 00:00 2026-02-20T11:28:30.280 INFO:teuthology.orchestra.run.trial072.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 16 MB/s | 358 kB 00:00 2026-02-20T11:28:30.297 INFO:teuthology.orchestra.run.trial072.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 14 MB/s | 254 kB 00:00 2026-02-20T11:28:30.355 INFO:teuthology.orchestra.run.trial072.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 35 MB/s | 2.0 MB 00:00 2026-02-20T11:28:30.372 INFO:teuthology.orchestra.run.trial072.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 8.2 MB/s | 144 kB 00:00 2026-02-20T11:28:30.389 INFO:teuthology.orchestra.run.trial072.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 7.9 MB/s | 139 kB 00:00 2026-02-20T11:28:30.405 INFO:teuthology.orchestra.run.trial072.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 3.6 MB/s | 56 kB 00:00 2026-02-20T11:28:30.419 INFO:teuthology.orchestra.run.trial072.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 736 kB/s | 11 kB 00:00 2026-02-20T11:28:30.434 INFO:teuthology.orchestra.run.trial072.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 1.2 MB/s | 18 kB 00:00 2026-02-20T11:28:30.449 INFO:teuthology.orchestra.run.trial072.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 1.5 MB/s | 23 kB 00:00 2026-02-20T11:28:30.464 INFO:teuthology.orchestra.run.trial072.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 1.3 MB/s | 20 kB 00:00 2026-02-20T11:28:30.479 INFO:teuthology.orchestra.run.trial072.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 1.3 MB/s | 19 kB 00:00 2026-02-20T11:28:30.494 INFO:teuthology.orchestra.run.trial072.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 1.7 MB/s | 26 kB 00:00 2026-02-20T11:28:30.516 INFO:teuthology.orchestra.run.trial072.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 47 MB/s | 1.0 MB 00:00 2026-02-20T11:28:30.531 INFO:teuthology.orchestra.run.trial072.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 4.9 MB/s | 79 kB 00:00 2026-02-20T11:28:30.547 INFO:teuthology.orchestra.run.trial072.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 5.5 MB/s | 86 kB 00:00 2026-02-20T11:28:30.562 INFO:teuthology.orchestra.run.trial072.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 3.7 MB/s | 58 kB 00:00 2026-02-20T11:28:30.576 INFO:teuthology.orchestra.run.trial072.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 1.1 MB/s | 16 kB 00:00 2026-02-20T11:28:30.592 INFO:teuthology.orchestra.run.trial072.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 5.7 MB/s | 90 kB 00:00 2026-02-20T11:28:30.607 INFO:teuthology.orchestra.run.trial072.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 2.0 MB/s | 31 kB 00:00 2026-02-20T11:28:30.624 INFO:teuthology.orchestra.run.trial072.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 11 MB/s | 188 kB 00:00 2026-02-20T11:28:30.639 INFO:teuthology.orchestra.run.trial072.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 3.8 MB/s | 59 kB 00:00 2026-02-20T11:28:30.655 INFO:teuthology.orchestra.run.trial072.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 7.7 MB/s | 125 kB 00:00 2026-02-20T11:28:30.670 INFO:teuthology.orchestra.run.trial072.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 2.3 MB/s | 36 kB 00:00 2026-02-20T11:28:30.685 INFO:teuthology.orchestra.run.trial072.stdout:(90/96): python3-typing-extensions-4.15.0-1.el9 5.5 MB/s | 86 kB 00:00 2026-02-20T11:28:30.700 INFO:teuthology.orchestra.run.trial072.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 5.9 MB/s | 90 kB 00:00 2026-02-20T11:28:30.716 INFO:teuthology.orchestra.run.trial072.stdout:(92/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 3.1 MB/s | 48 kB 00:00 2026-02-20T11:28:30.731 INFO:teuthology.orchestra.run.trial072.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 1.5 MB/s | 22 kB 00:00 2026-02-20T11:28:30.745 INFO:teuthology.orchestra.run.trial072.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 1.3 MB/s | 20 kB 00:00 2026-02-20T11:28:45.924 INFO:teuthology.orchestra.run.trial072.stdout:(95/96): ceph-mgr-dashboard-20.3.0-5357.g0c3148 2.3 MB/s | 51 MB 00:22 2026-02-20T11:28:59.630 INFO:teuthology.orchestra.run.trial072.stdout:(96/96): ceph-osd-crimson-20.3.0-5357.g0c3148ca 2.4 MB/s | 88 MB 00:37 2026-02-20T11:28:59.631 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:28:59.632 INFO:teuthology.orchestra.run.trial072.stdout:Total 5.8 MB/s | 226 MB 00:38 2026-02-20T11:29:00.029 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:29:00.069 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:29:00.070 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:29:00.869 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:29:00.870 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:29:01.708 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:29:01.710 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-20T11:29:01.713 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-20T11:29:01.735 INFO:teuthology.orchestra.run.trial072.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-20T11:29:01.768 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-20T11:29:01.774 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-20T11:29:01.777 INFO:teuthology.orchestra.run.trial072.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-20T11:29:01.781 INFO:teuthology.orchestra.run.trial072.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-20T11:29:01.804 INFO:teuthology.orchestra.run.trial072.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-20T11:29:01.817 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-20T11:29:01.820 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-20T11:29:01.826 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-20T11:29:01.828 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-20T11:29:01.839 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-20T11:29:01.845 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-20T11:29:01.870 INFO:teuthology.orchestra.run.trial072.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-20T11:29:01.873 INFO:teuthology.orchestra.run.trial072.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-20T11:29:01.923 INFO:teuthology.orchestra.run.trial072.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-20T11:29:01.935 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-20T11:29:01.938 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-20T11:29:01.942 INFO:teuthology.orchestra.run.trial072.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-20T11:29:01.947 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-20T11:29:01.952 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-20T11:29:01.955 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-20T11:29:01.970 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-20T11:29:01.974 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-20T11:29:01.979 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-20T11:29:01.990 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-20T11:29:01.999 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-20T11:29:02.006 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-20T11:29:02.025 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-20T11:29:02.036 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-20T11:29:02.052 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-20T11:29:02.056 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-20T11:29:02.064 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-20T11:29:02.104 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-20T11:29:02.427 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-20T11:29:02.437 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-20T11:29:02.441 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-20T11:29:02.447 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-20T11:29:02.449 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-20T11:29:02.477 INFO:teuthology.orchestra.run.trial072.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-20T11:29:02.520 INFO:teuthology.orchestra.run.trial072.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-20T11:29:02.529 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-20T11:29:02.535 INFO:teuthology.orchestra.run.trial072.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-20T11:29:02.538 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-20T11:29:02.544 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-20T11:29:02.548 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-20T11:29:02.554 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-20T11:29:02.558 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-20T11:29:02.586 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-20T11:29:02.596 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-20T11:29:02.601 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-20T11:29:02.608 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-20T11:29:02.643 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-20T11:29:02.867 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-20T11:29:02.890 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-20T11:29:02.894 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-20T11:29:02.955 INFO:teuthology.orchestra.run.trial072.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-20T11:29:02.956 INFO:teuthology.orchestra.run.trial072.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-20T11:29:02.978 INFO:teuthology.orchestra.run.trial072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-20T11:29:03.302 INFO:teuthology.orchestra.run.trial072.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-20T11:29:03.376 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-20T11:29:04.052 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-20T11:29:04.069 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-20T11:29:04.079 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-20T11:29:04.085 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-20T11:29:04.099 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-20T11:29:04.115 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-20T11:29:04.191 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-20T11:29:04.199 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-20T11:29:04.223 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-20T11:29:04.253 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-20T11:29:04.304 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-20T11:29:04.311 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-20T11:29:04.314 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-20T11:29:04.315 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-20T11:29:04.334 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:29:04.334 INFO:teuthology.orchestra.run.trial072.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T11:29:04.334 INFO:teuthology.orchestra.run.trial072.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T11:29:04.335 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:04.343 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:29:04.369 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:29:04.370 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T11:29:04.370 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:04.404 INFO:teuthology.orchestra.run.trial072.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-20T11:29:04.992 INFO:teuthology.orchestra.run.trial072.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-20T11:29:10.395 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 80/96 2026-02-20T11:29:10.458 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 81/96 2026-02-20T11:29:10.461 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 82/96 2026-02-20T11:29:10.475 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 82/96 2026-02-20T11:29:10.475 INFO:teuthology.orchestra.run.trial072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:29:10.476 INFO:teuthology.orchestra.run.trial072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T11:29:10.476 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:10.486 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 83/96 2026-02-20T11:29:10.511 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 83/96 2026-02-20T11:29:10.511 INFO:teuthology.orchestra.run.trial072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:29:10.511 INFO:teuthology.orchestra.run.trial072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T11:29:10.511 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T11:29:10.512 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T11:29:10.512 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:10.572 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 84/96 2026-02-20T11:29:10.574 INFO:teuthology.orchestra.run.trial072.stdout: Installing : cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 84/96 2026-02-20T11:29:10.583 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9 85/96 2026-02-20T11:29:10.607 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el 86/96 2026-02-20T11:29:10.608 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 87/96 2026-02-20T11:29:13.504 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 87/96 2026-02-20T11:29:13.510 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 88/96 2026-02-20T11:29:13.964 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 88/96 2026-02-20T11:29:13.965 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 89/96 2026-02-20T11:29:13.977 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 89/96 2026-02-20T11:29:13.978 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 90/96 2026-02-20T11:29:14.041 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 90/96 2026-02-20T11:29:14.082 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9. 91/96 2026-02-20T11:29:14.083 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 92/96 2026-02-20T11:29:14.104 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 92/96 2026-02-20T11:29:14.105 INFO:teuthology.orchestra.run.trial072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:29:14.105 INFO:teuthology.orchestra.run.trial072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T11:29:14.105 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T11:29:14.105 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T11:29:14.105 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:14.116 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 93/96 2026-02-20T11:29:14.332 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 93/96 2026-02-20T11:29:14.334 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 94/96 2026-02-20T11:29:14.355 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 94/96 2026-02-20T11:29:14.355 INFO:teuthology.orchestra.run.trial072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:29:14.355 INFO:teuthology.orchestra.run.trial072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T11:29:14.355 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T11:29:14.355 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T11:29:14.355 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:14.438 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 95/96 2026-02-20T11:29:14.459 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 95/96 2026-02-20T11:29:14.459 INFO:teuthology.orchestra.run.trial072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:29:14.459 INFO:teuthology.orchestra.run.trial072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T11:29:14.459 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T11:29:14.459 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T11:29:14.459 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:14.460 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 96/96 2026-02-20T11:29:14.467 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 96/96 2026-02-20T11:29:14.491 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 96/96 2026-02-20T11:29:15.952 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 96/96 2026-02-20T11:29:15.952 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/96 2026-02-20T11:29:15.952 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 5/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 6/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 7/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el 8/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 9/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 10/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 11/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 12/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9. 13/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 14/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9 15/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 16/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 17/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-20T11:29:15.953 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-20T11:29:15.954 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-20T11:29:15.955 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-20T11:29:15.956 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-20T11:29:15.957 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-20T11:29:15.957 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-20T11:29:15.957 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:16.020 INFO:teuthology.orchestra.run.trial072.stdout: ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T11:29:16.021 INFO:teuthology.orchestra.run.trial072.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-20T11:29:16.022 INFO:teuthology.orchestra.run.trial072.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-20T11:29:16.023 INFO:teuthology.orchestra.run.trial072.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T11:29:16.024 INFO:teuthology.orchestra.run.trial072.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: zip-3.0-35.el9.x86_64 2026-02-20T11:29:16.025 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:16.026 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:16.087 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-base 2026-02-20T11:29:16.401 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:34 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:16.467 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:16.485 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:16.486 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:16.486 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:16.507 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install cephadm 2026-02-20T11:29:16.819 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:34 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:16.884 INFO:teuthology.orchestra.run.trial072.stdout:Package cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:29:16.902 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:16.903 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:16.903 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:16.923 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T11:29:17.234 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:35 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:17.318 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repo Size 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 149 k 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:Install 1 Package 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 149 k 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 455 k 2026-02-20T11:29:17.319 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:29:17.787 INFO:teuthology.orchestra.run.trial072.stdout:ceph-immutable-object-cache-20.3.0-5357.g0c3148 319 kB/s | 149 kB 00:00 2026-02-20T11:29:17.788 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:29:17.788 INFO:teuthology.orchestra.run.trial072.stdout:Total 318 kB/s | 149 kB 00:00 2026-02-20T11:29:17.788 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:29:17.793 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:29:17.793 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:29:17.824 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:29:17.824 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:29:17.879 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:29:17.881 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:29:17.902 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:29:17.903 INFO:teuthology.orchestra.run.trial072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:29:17.903 INFO:teuthology.orchestra.run.trial072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T11:29:17.903 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:18.210 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:29:18.210 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:18.211 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:29:18.211 INFO:teuthology.orchestra.run.trial072.stdout: ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:18.211 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:18.211 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:18.237 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr 2026-02-20T11:29:18.555 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:36 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:18.620 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:18.638 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:18.639 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:18.639 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:18.660 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T11:29:18.971 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:36 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:19.036 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:29:19.055 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:19.056 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:19.056 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:19.077 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T11:29:19.392 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:37 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:19.458 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:29:19.476 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:19.477 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:19.477 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:19.498 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr-rook 2026-02-20T11:29:19.808 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:37 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:19.876 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:29:19.894 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:19.895 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:19.895 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:19.916 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T11:29:20.234 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:38 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:20.298 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:29:20.317 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:20.317 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:20.317 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:20.339 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-osd 2026-02-20T11:29:20.651 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:38 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:20.716 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:20.734 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:20.735 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:20.735 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:20.756 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-osd-classic 2026-02-20T11:29:21.069 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:39 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:21.134 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:21.152 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:21.153 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:21.153 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:21.174 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-fuse 2026-02-20T11:29:21.491 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:39 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:21.574 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repo Size 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout: ceph-fuse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 914 k 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:21.575 INFO:teuthology.orchestra.run.trial072.stdout:Install 1 Package 2026-02-20T11:29:21.576 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:21.576 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 914 k 2026-02-20T11:29:21.576 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 2.9 M 2026-02-20T11:29:21.576 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:29:22.397 INFO:teuthology.orchestra.run.trial072.stdout:ceph-fuse-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 1.1 MB/s | 914 kB 00:00 2026-02-20T11:29:22.397 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:29:22.397 INFO:teuthology.orchestra.run.trial072.stdout:Total 1.1 MB/s | 914 kB 00:00 2026-02-20T11:29:22.397 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:29:22.403 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:29:22.403 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:29:22.436 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:29:22.437 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:29:22.526 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:29:22.544 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:22.783 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:22.841 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:22.841 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:22.842 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:29:22.842 INFO:teuthology.orchestra.run.trial072.stdout: ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:22.842 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:22.842 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:22.868 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-volume 2026-02-20T11:29:23.183 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:41 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:23.250 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:29:23.268 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:23.269 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:23.269 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:23.291 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install librados-devel 2026-02-20T11:29:23.611 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:41 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repo Size 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout: librados-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 124 k 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:Install 1 Package 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 124 k 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 434 k 2026-02-20T11:29:23.694 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:29:24.172 INFO:teuthology.orchestra.run.trial072.stdout:librados-devel-20.3.0-5357.g0c3148ca.el9.x86_64 259 kB/s | 124 kB 00:00 2026-02-20T11:29:24.173 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:29:24.173 INFO:teuthology.orchestra.run.trial072.stdout:Total 259 kB/s | 124 kB 00:00 2026-02-20T11:29:24.173 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:29:24.176 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:29:24.176 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:29:24.211 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:29:24.211 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:29:24.267 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:29:24.284 INFO:teuthology.orchestra.run.trial072.stdout: Installing : librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:24.371 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:24.430 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:24.431 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:24.431 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:29:24.431 INFO:teuthology.orchestra.run.trial072.stdout: librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:24.431 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:24.431 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:24.460 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install libcephfs2 2026-02-20T11:29:24.783 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:42 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:24.850 INFO:teuthology.orchestra.run.trial072.stdout:Package libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:24.869 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:24.869 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:24.869 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:24.891 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install libcephfs-devel 2026-02-20T11:29:25.201 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:43 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repo Size 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout: libcephfs-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 37 k 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:29:25.284 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:25.285 INFO:teuthology.orchestra.run.trial072.stdout:Install 1 Package 2026-02-20T11:29:25.285 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:25.285 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 37 k 2026-02-20T11:29:25.285 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 171 k 2026-02-20T11:29:25.285 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:29:25.638 INFO:teuthology.orchestra.run.trial072.stdout:libcephfs-devel-20.3.0-5357.g0c3148ca.el9.x86_6 106 kB/s | 37 kB 00:00 2026-02-20T11:29:25.639 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:29:25.639 INFO:teuthology.orchestra.run.trial072.stdout:Total 105 kB/s | 37 kB 00:00 2026-02-20T11:29:25.639 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:29:25.641 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:29:25.641 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:29:25.644 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:29:25.644 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:29:25.666 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:29:25.682 INFO:teuthology.orchestra.run.trial072.stdout: Installing : libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:25.740 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:25.801 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:25.802 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:25.802 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:29:25.802 INFO:teuthology.orchestra.run.trial072.stdout: libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:25.802 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:25.802 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:25.830 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install librados2 2026-02-20T11:29:26.142 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:44 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:26.208 INFO:teuthology.orchestra.run.trial072.stdout:Package librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:26.227 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:26.228 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:26.228 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:26.249 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install librbd1 2026-02-20T11:29:26.556 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:44 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:26.623 INFO:teuthology.orchestra.run.trial072.stdout:Package librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:26.641 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:26.642 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:26.642 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:26.663 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install python3-rados 2026-02-20T11:29:26.978 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:44 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:27.045 INFO:teuthology.orchestra.run.trial072.stdout:Package python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:27.064 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:27.064 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:27.064 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:27.086 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install python3-rgw 2026-02-20T11:29:27.403 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:45 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:27.472 INFO:teuthology.orchestra.run.trial072.stdout:Package python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:27.491 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:27.492 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:27.492 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:27.513 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install python3-cephfs 2026-02-20T11:29:27.828 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:45 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:27.895 INFO:teuthology.orchestra.run.trial072.stdout:Package python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:27.914 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:27.914 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:27.914 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:27.935 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install python3-rbd 2026-02-20T11:29:28.247 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:46 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:28.313 INFO:teuthology.orchestra.run.trial072.stdout:Package python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:28.331 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:28.332 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:28.332 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:28.354 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install rbd-fuse 2026-02-20T11:29:28.673 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:46 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:28.757 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:28.757 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:28.757 INFO:teuthology.orchestra.run.trial072.stdout: Package Architecture Version Repository Size 2026-02-20T11:29:28.757 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:28.757 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:29:28.757 INFO:teuthology.orchestra.run.trial072.stdout: rbd-fuse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 86 k 2026-02-20T11:29:28.757 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:28.757 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:29:28.758 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:28.758 INFO:teuthology.orchestra.run.trial072.stdout:Install 1 Package 2026-02-20T11:29:28.758 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:28.758 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 86 k 2026-02-20T11:29:28.758 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 236 k 2026-02-20T11:29:28.758 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:29:29.176 INFO:teuthology.orchestra.run.trial072.stdout:rbd-fuse-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 207 kB/s | 86 kB 00:00 2026-02-20T11:29:29.176 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:29:29.176 INFO:teuthology.orchestra.run.trial072.stdout:Total 206 kB/s | 86 kB 00:00 2026-02-20T11:29:29.176 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:29:29.180 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:29:29.180 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:29:29.211 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:29:29.211 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:29:29.262 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:29:29.279 INFO:teuthology.orchestra.run.trial072.stdout: Installing : rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:29.363 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:29.422 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:29.422 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:29.422 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:29:29.422 INFO:teuthology.orchestra.run.trial072.stdout: rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:29.422 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:29.422 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:29.449 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install rbd-mirror 2026-02-20T11:29:29.761 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:47 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:29.844 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repo Size 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout: rbd-mirror x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.6 M 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:Install 1 Package 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 2.6 M 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 11 M 2026-02-20T11:29:29.845 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:29:30.718 INFO:teuthology.orchestra.run.trial072.stdout:rbd-mirror-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 3.0 MB/s | 2.6 MB 00:00 2026-02-20T11:29:30.719 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:29:30.719 INFO:teuthology.orchestra.run.trial072.stdout:Total 3.0 MB/s | 2.6 MB 00:00 2026-02-20T11:29:30.719 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:29:30.724 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:29:30.724 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:29:30.763 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:29:30.763 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:29:30.925 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:29:30.927 INFO:teuthology.orchestra.run.trial072.stdout: Installing : rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:30.949 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:30.949 INFO:teuthology.orchestra.run.trial072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:29:30.949 INFO:teuthology.orchestra.run.trial072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T11:29:30.950 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T11:29:30.950 INFO:teuthology.orchestra.run.trial072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T11:29:30.950 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:31.251 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:31.251 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:31.251 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:29:31.251 INFO:teuthology.orchestra.run.trial072.stdout: rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:31.252 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:31.252 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:31.281 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install rbd-nbd 2026-02-20T11:29:31.595 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:49 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:31.679 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:31.680 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:31.680 INFO:teuthology.orchestra.run.trial072.stdout: Package Architecture Version Repository Size 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout: rbd-nbd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 167 k 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout:Install 1 Package 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 167 k 2026-02-20T11:29:31.681 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 496 k 2026-02-20T11:29:31.682 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:29:32.122 INFO:teuthology.orchestra.run.trial072.stdout:rbd-nbd-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 377 kB/s | 167 kB 00:00 2026-02-20T11:29:32.122 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:29:32.122 INFO:teuthology.orchestra.run.trial072.stdout:Total 377 kB/s | 167 kB 00:00 2026-02-20T11:29:32.122 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:29:32.127 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:29:32.127 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:29:32.161 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:29:32.161 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:29:32.220 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:29:32.237 INFO:teuthology.orchestra.run.trial072.stdout: Installing : rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:32.322 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:32.381 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:29:32.381 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:32.381 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:29:32.381 INFO:teuthology.orchestra.run.trial072.stdout: rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:29:32.381 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:32.381 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:32.410 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-osd-crimson 2026-02-20T11:29:32.732 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:50 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:32.798 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:29:32.817 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:32.817 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:29:32.817 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:29:32.839 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-20T11:29:33.157 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:02:51 ago on Fri 20 Feb 2026 11:26:42 AM UTC. 2026-02-20T11:29:33.242 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:29:33.242 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout: Package Arch Version Repo Size 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout:Installing: 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 433 M 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout:Installing weak dependencies: 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout: ceph-debugsource x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 12 M 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout:Transaction Summary 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout:================================================================================ 2026-02-20T11:29:33.243 INFO:teuthology.orchestra.run.trial072.stdout:Install 2 Packages 2026-02-20T11:29:33.244 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:29:33.244 INFO:teuthology.orchestra.run.trial072.stdout:Total download size: 445 M 2026-02-20T11:29:33.244 INFO:teuthology.orchestra.run.trial072.stdout:Installed size: 2.7 G 2026-02-20T11:29:33.244 INFO:teuthology.orchestra.run.trial072.stdout:Downloading Packages: 2026-02-20T11:29:39.416 INFO:teuthology.orchestra.run.trial072.stdout:(1/2): ceph-debugsource-20.3.0-5357.g0c3148ca.e 2.0 MB/s | 12 MB 00:06 2026-02-20T11:34:32.873 INFO:teuthology.orchestra.run.trial072.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5357.g 1.4 MB/s | 433 MB 04:59 2026-02-20T11:34:32.874 INFO:teuthology.orchestra.run.trial072.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:34:32.875 INFO:teuthology.orchestra.run.trial072.stdout:Total 1.5 MB/s | 445 MB 04:59 2026-02-20T11:34:32.875 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction check 2026-02-20T11:34:32.880 INFO:teuthology.orchestra.run.trial072.stdout:Transaction check succeeded. 2026-02-20T11:34:32.880 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction test 2026-02-20T11:34:34.260 INFO:teuthology.orchestra.run.trial072.stdout:Transaction test succeeded. 2026-02-20T11:34:34.261 INFO:teuthology.orchestra.run.trial072.stdout:Running transaction 2026-02-20T11:34:37.873 INFO:teuthology.orchestra.run.trial072.stdout: Preparing : 1/1 2026-02-20T11:35:00.892 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/2 2026-02-20T11:35:00.910 INFO:teuthology.orchestra.run.trial072.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:35:02.933 INFO:teuthology.orchestra.run.trial072.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:35:02.933 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/2 2026-02-20T11:35:02.992 INFO:teuthology.orchestra.run.trial072.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:35:02.992 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:02.992 INFO:teuthology.orchestra.run.trial072.stdout:Installed: 2026-02-20T11:35:02.992 INFO:teuthology.orchestra.run.trial072.stdout: ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:02.993 INFO:teuthology.orchestra.run.trial072.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:35:02.993 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:02.993 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:03.071 DEBUG:teuthology.parallel:result is None 2026-02-20T11:35:03.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:35:03.237 DEBUG:teuthology.orchestra.run.trial072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T11:35:03.258 INFO:teuthology.orchestra.run.trial072.stdout:20.3.0-5357.g0c3148ca.el9 2026-02-20T11:35:03.259 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5357.g0c3148ca.el9 2026-02-20T11:35:03.259 INFO:teuthology.task.install:The correct ceph version 20.3.0-5357.g0c3148ca is installed. 2026-02-20T11:35:03.261 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-20T11:35:03.261 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:03.262 DEBUG:teuthology.orchestra.run.trial072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-20T11:35:03.324 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-20T11:35:03.324 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:03.324 DEBUG:teuthology.orchestra.run.trial072:> sudo dd of=/usr/bin/daemon-helper 2026-02-20T11:35:03.387 DEBUG:teuthology.orchestra.run.trial072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-20T11:35:03.452 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-20T11:35:03.453 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:03.453 DEBUG:teuthology.orchestra.run.trial072:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-20T11:35:03.515 DEBUG:teuthology.orchestra.run.trial072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-20T11:35:03.580 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-20T11:35:03.581 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:03.581 DEBUG:teuthology.orchestra.run.trial072:> sudo dd of=/usr/bin/stdin-killer 2026-02-20T11:35:03.645 DEBUG:teuthology.orchestra.run.trial072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-20T11:35:03.710 INFO:teuthology.run_tasks:Running task pexec... 2026-02-20T11:35:03.727 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-20T11:35:03.727 DEBUG:teuthology.orchestra.run.trial072:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-20T11:35:03.753 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial072.front.sepia.ceph.com 2026-02-20T11:35:03.753 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-20T11:35:03.804 DEBUG:teuthology.parallel:result is None 2026-02-20T11:35:03.804 INFO:teuthology.run_tasks:Running task install... 2026-02-20T11:35:03.818 DEBUG:teuthology.task.install:project ceph 2026-02-20T11:35:03.818 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}} 2026-02-20T11:35:03.818 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'} 2026-02-20T11:35:03.819 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T11:35:03.827 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T11:35:03.827 INFO:teuthology.task.install:extra packages: [] 2026-02-20T11:35:03.827 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'tag': None, 'wait_for_package': False} 2026-02-20T11:35:03.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:35:03.987 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/ 2026-02-20T11:35:03.987 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5357.g0c3148ca 2026-02-20T11:35:04.378 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T11:35:04.378 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:04.378 DEBUG:teuthology.orchestra.run.trial072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T11:35:04.403 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-20T11:35:04.403 DEBUG:teuthology.orchestra.run.trial072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T11:35:04.468 DEBUG:teuthology.orchestra.run.trial072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T11:35:04.533 DEBUG:teuthology.orchestra.run.trial072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T11:35:04.591 INFO:teuthology.orchestra.run.trial072.stdout:check_obsoletes = 1 2026-02-20T11:35:04.600 DEBUG:teuthology.orchestra.run.trial072:> sudo yum clean all 2026-02-20T11:35:04.762 INFO:teuthology.orchestra.run.trial072.stdout:56 files removed 2026-02-20T11:35:04.778 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-radosgw 2026-02-20T11:35:05.638 INFO:teuthology.orchestra.run.trial072.stdout:ceph packages for x86_64 133 kB/s | 94 kB 00:00 2026-02-20T11:35:06.289 INFO:teuthology.orchestra.run.trial072.stdout:ceph noarch packages 39 kB/s | 24 kB 00:00 2026-02-20T11:35:06.858 INFO:teuthology.orchestra.run.trial072.stdout:ceph source packages 3.8 kB/s | 2.1 kB 00:00 2026-02-20T11:35:08.126 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.9 MB 00:01 2026-02-20T11:35:11.164 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-20T11:35:14.646 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 7.8 MB 00:01 2026-02-20T11:35:15.555 INFO:teuthology.orchestra.run.trial072.stdout:CentOS Stream 9 - Extras packages 109 kB/s | 20 kB 00:00 2026-02-20T11:35:27.423 INFO:teuthology.orchestra.run.trial072.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 20 MB 00:11 2026-02-20T11:35:30.689 INFO:teuthology.orchestra.run.trial072.stdout:lab-extras 459 kB/s | 50 kB 00:00 2026-02-20T11:35:31.876 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:31.895 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:31.896 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:31.896 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:31.919 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-test 2026-02-20T11:35:32.233 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:32.298 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:32.316 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:32.317 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:32.318 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:32.339 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph 2026-02-20T11:35:32.657 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:32.723 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:32.742 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:32.743 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:32.743 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:32.765 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-base 2026-02-20T11:35:33.076 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:33.141 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:33.159 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:33.160 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:33.160 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:33.182 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install cephadm 2026-02-20T11:35:33.492 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:33.558 INFO:teuthology.orchestra.run.trial072.stdout:Package cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:35:33.576 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:33.577 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:33.577 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:33.598 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T11:35:33.914 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:33.980 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:33.998 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:33.999 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:33.999 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:34.021 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr 2026-02-20T11:35:34.330 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:34.395 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:34.414 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:34.415 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:34.415 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:34.436 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T11:35:34.752 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:34.819 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:35:34.837 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:34.838 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:34.838 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:34.859 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T11:35:35.180 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:35.248 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:35:35.266 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:35.267 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:35.267 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:35.290 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr-rook 2026-02-20T11:35:35.608 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:35.674 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:35:35.693 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:35.693 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:35.694 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:35.715 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T11:35:36.033 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:36.099 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:35:36.118 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:36.119 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:36.119 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:36.141 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-osd 2026-02-20T11:35:36.454 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:36.518 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:36.536 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:36.537 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:36.537 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:36.559 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-osd-classic 2026-02-20T11:35:36.880 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:36.946 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:36.964 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:36.965 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:36.965 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:36.987 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-fuse 2026-02-20T11:35:37.300 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:37.366 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:37.385 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:37.385 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:37.386 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:37.408 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install ceph-volume 2026-02-20T11:35:37.722 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:37.788 INFO:teuthology.orchestra.run.trial072.stdout:Package ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:35:37.808 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:37.809 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:37.809 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:37.831 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install librados-devel 2026-02-20T11:35:38.145 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:38.211 INFO:teuthology.orchestra.run.trial072.stdout:Package librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:38.230 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:38.230 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:38.231 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:38.253 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install libcephfs2 2026-02-20T11:35:38.568 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:38.634 INFO:teuthology.orchestra.run.trial072.stdout:Package libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:38.652 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:38.653 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:38.653 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:38.675 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install libcephfs-devel 2026-02-20T11:35:38.990 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:39.057 INFO:teuthology.orchestra.run.trial072.stdout:Package libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:39.075 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:39.076 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:39.076 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:39.098 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install librados2 2026-02-20T11:35:39.412 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:39.477 INFO:teuthology.orchestra.run.trial072.stdout:Package librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:39.496 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:39.496 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:39.496 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:39.517 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install librbd1 2026-02-20T11:35:39.828 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:39.894 INFO:teuthology.orchestra.run.trial072.stdout:Package librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:39.912 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:39.913 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:39.913 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:39.934 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install python3-rados 2026-02-20T11:35:40.246 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:40.311 INFO:teuthology.orchestra.run.trial072.stdout:Package python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:40.329 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:40.330 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:40.330 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:40.351 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install python3-rgw 2026-02-20T11:35:40.671 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:40.736 INFO:teuthology.orchestra.run.trial072.stdout:Package python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:40.755 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:40.756 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:40.756 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:40.777 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install python3-cephfs 2026-02-20T11:35:41.095 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:41.162 INFO:teuthology.orchestra.run.trial072.stdout:Package python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:41.181 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:41.181 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:41.182 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:41.203 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install python3-rbd 2026-02-20T11:35:41.509 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:41.574 INFO:teuthology.orchestra.run.trial072.stdout:Package python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:41.592 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:41.593 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:41.593 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:41.614 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install rbd-fuse 2026-02-20T11:35:41.929 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:41.996 INFO:teuthology.orchestra.run.trial072.stdout:Package rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:42.015 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:42.015 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:42.015 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:42.037 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install rbd-mirror 2026-02-20T11:35:42.346 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:42.412 INFO:teuthology.orchestra.run.trial072.stdout:Package rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:42.432 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:42.432 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:42.432 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:42.454 DEBUG:teuthology.orchestra.run.trial072:> sudo yum -y install rbd-nbd 2026-02-20T11:35:42.764 INFO:teuthology.orchestra.run.trial072.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 11:35:30 AM UTC. 2026-02-20T11:35:42.830 INFO:teuthology.orchestra.run.trial072.stdout:Package rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:35:42.848 INFO:teuthology.orchestra.run.trial072.stdout:Dependencies resolved. 2026-02-20T11:35:42.849 INFO:teuthology.orchestra.run.trial072.stdout:Nothing to do. 2026-02-20T11:35:42.849 INFO:teuthology.orchestra.run.trial072.stdout:Complete! 2026-02-20T11:35:42.871 DEBUG:teuthology.parallel:result is None 2026-02-20T11:35:42.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:35:43.111 DEBUG:teuthology.orchestra.run.trial072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T11:35:43.132 INFO:teuthology.orchestra.run.trial072.stdout:20.3.0-5357.g0c3148ca.el9 2026-02-20T11:35:43.132 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5357.g0c3148ca.el9 2026-02-20T11:35:43.132 INFO:teuthology.task.install:The correct ceph version 20.3.0-5357.g0c3148ca is installed. 2026-02-20T11:35:43.135 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-20T11:35:43.135 INFO:teuthology.run_tasks:Running task ceph... 2026-02-20T11:35:43.232 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-20T11:35:43.232 DEBUG:teuthology.orchestra.run.trial072:> sudo chmod 777 /var/log/ceph 2026-02-20T11:35:43.258 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-20T11:35:43.258 DEBUG:teuthology.orchestra.run.trial072:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-20T11:35:43.324 INFO:tasks.ceph:Creating extra log directories... 2026-02-20T11:35:43.324 DEBUG:teuthology.orchestra.run.trial072:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-20T11:35:43.399 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-20T11:35:43.399 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(POOL_FULL\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-20T11:35:43.399 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60522', 'branch': 'wip-perezjos-crimson-only-19-02-2026-PR64439', 'description': 'crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_api_tests}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '60522', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-perezjos-crimson-only-19-02-2026-PR64439'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(POOL_FULL\\)'], 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '445d5c9ba18a4040e70e25303230c36c77773acb'}}, 'owner': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'priority': 101, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 1736, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '445d5c9ba18a4040e70e25303230c36c77773acb', 'targets': {'trial072.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOMpdyf701I1KsmqXu/xjfUFX9btE4RLcSi9jddRmtpfMuLLSpLP1swn2ZraA6jVP6hLSMyFL0lvKVvEaxqE2JE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(POOL_FULL\\)'], 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rbd/test_librbd.sh']}, 'env': {'RBD_FEATURES': '61'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'reimage-ssh-down', 'teuthology_sha1': 'e6d9099ce03dc21193fda653159ec82465c3a94a', 'timestamp': '2026-02-20_10:21:31', 'tube': 'trial', 'user': 'jjperez', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-20T11:35:43.400 DEBUG:teuthology.orchestra.run.trial072:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-20T11:35:43.445 DEBUG:teuthology.orchestra.run.trial072:> sudo install -d -m0777 -- /var/run/ceph 2026-02-20T11:35:43.509 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:43.510 DEBUG:teuthology.orchestra.run.trial072:> dd if=/scratch_devs of=/dev/stdout 2026-02-20T11:35:43.567 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-20T11:35:43.567 DEBUG:teuthology.orchestra.run.trial072:> stat /dev/vg_nvme/lv_1 2026-02-20T11:35:43.625 INFO:teuthology.orchestra.run.trial072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-20T11:35:43.626 INFO:teuthology.orchestra.run.trial072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:35:43.626 INFO:teuthology.orchestra.run.trial072.stdout:Device: 6h/6d Inode: 1298 Links: 1 2026-02-20T11:35:43.626 INFO:teuthology.orchestra.run.trial072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:35:43.626 INFO:teuthology.orchestra.run.trial072.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:35:43.626 INFO:teuthology.orchestra.run.trial072.stdout:Access: 2026-02-20 11:29:31.096415185 +0000 2026-02-20T11:35:43.626 INFO:teuthology.orchestra.run.trial072.stdout:Modify: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:43.626 INFO:teuthology.orchestra.run.trial072.stdout:Change: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:43.626 INFO:teuthology.orchestra.run.trial072.stdout: Birth: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:43.627 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-20T11:35:43.688 INFO:teuthology.orchestra.run.trial072.stderr:1+0 records in 2026-02-20T11:35:43.688 INFO:teuthology.orchestra.run.trial072.stderr:1+0 records out 2026-02-20T11:35:43.689 INFO:teuthology.orchestra.run.trial072.stderr:512 bytes copied, 0.000147746 s, 3.5 MB/s 2026-02-20T11:35:43.690 DEBUG:teuthology.orchestra.run.trial072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-20T11:35:43.749 DEBUG:teuthology.orchestra.run.trial072:> stat /dev/vg_nvme/lv_2 2026-02-20T11:35:43.804 INFO:teuthology.orchestra.run.trial072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-20T11:35:43.804 INFO:teuthology.orchestra.run.trial072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:35:43.804 INFO:teuthology.orchestra.run.trial072.stdout:Device: 6h/6d Inode: 1302 Links: 1 2026-02-20T11:35:43.804 INFO:teuthology.orchestra.run.trial072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:35:43.804 INFO:teuthology.orchestra.run.trial072.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:35:43.805 INFO:teuthology.orchestra.run.trial072.stdout:Access: 2026-02-20 11:29:31.096415185 +0000 2026-02-20T11:35:43.805 INFO:teuthology.orchestra.run.trial072.stdout:Modify: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:43.805 INFO:teuthology.orchestra.run.trial072.stdout:Change: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:43.805 INFO:teuthology.orchestra.run.trial072.stdout: Birth: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:43.805 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-20T11:35:43.867 INFO:teuthology.orchestra.run.trial072.stderr:1+0 records in 2026-02-20T11:35:43.867 INFO:teuthology.orchestra.run.trial072.stderr:1+0 records out 2026-02-20T11:35:43.867 INFO:teuthology.orchestra.run.trial072.stderr:512 bytes copied, 0.000146543 s, 3.5 MB/s 2026-02-20T11:35:43.868 DEBUG:teuthology.orchestra.run.trial072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-20T11:35:43.925 DEBUG:teuthology.orchestra.run.trial072:> stat /dev/vg_nvme/lv_3 2026-02-20T11:35:43.981 INFO:teuthology.orchestra.run.trial072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-20T11:35:43.982 INFO:teuthology.orchestra.run.trial072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:35:43.982 INFO:teuthology.orchestra.run.trial072.stdout:Device: 6h/6d Inode: 1318 Links: 1 2026-02-20T11:35:43.982 INFO:teuthology.orchestra.run.trial072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:35:43.982 INFO:teuthology.orchestra.run.trial072.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:35:43.982 INFO:teuthology.orchestra.run.trial072.stdout:Access: 2026-02-20 11:29:31.096415185 +0000 2026-02-20T11:35:43.982 INFO:teuthology.orchestra.run.trial072.stdout:Modify: 2026-02-20 11:26:55.712649868 +0000 2026-02-20T11:35:43.982 INFO:teuthology.orchestra.run.trial072.stdout:Change: 2026-02-20 11:26:55.712649868 +0000 2026-02-20T11:35:43.982 INFO:teuthology.orchestra.run.trial072.stdout: Birth: 2026-02-20 11:26:55.712649868 +0000 2026-02-20T11:35:43.983 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-20T11:35:44.044 INFO:teuthology.orchestra.run.trial072.stderr:1+0 records in 2026-02-20T11:35:44.044 INFO:teuthology.orchestra.run.trial072.stderr:1+0 records out 2026-02-20T11:35:44.045 INFO:teuthology.orchestra.run.trial072.stderr:512 bytes copied, 0.000153547 s, 3.3 MB/s 2026-02-20T11:35:44.046 DEBUG:teuthology.orchestra.run.trial072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-20T11:35:44.103 DEBUG:teuthology.orchestra.run.trial072:> stat /dev/vg_nvme/lv_4 2026-02-20T11:35:44.160 INFO:teuthology.orchestra.run.trial072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-20T11:35:44.160 INFO:teuthology.orchestra.run.trial072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:35:44.160 INFO:teuthology.orchestra.run.trial072.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-20T11:35:44.160 INFO:teuthology.orchestra.run.trial072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:35:44.160 INFO:teuthology.orchestra.run.trial072.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:35:44.161 INFO:teuthology.orchestra.run.trial072.stdout:Access: 2026-02-20 11:29:31.097415146 +0000 2026-02-20T11:35:44.161 INFO:teuthology.orchestra.run.trial072.stdout:Modify: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:44.161 INFO:teuthology.orchestra.run.trial072.stdout:Change: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:44.161 INFO:teuthology.orchestra.run.trial072.stdout: Birth: 2026-02-20 11:26:55.711649906 +0000 2026-02-20T11:35:44.161 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-20T11:35:44.225 INFO:teuthology.orchestra.run.trial072.stderr:1+0 records in 2026-02-20T11:35:44.225 INFO:teuthology.orchestra.run.trial072.stderr:1+0 records out 2026-02-20T11:35:44.226 INFO:teuthology.orchestra.run.trial072.stderr:512 bytes copied, 0.000140903 s, 3.6 MB/s 2026-02-20T11:35:44.227 DEBUG:teuthology.orchestra.run.trial072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-20T11:35:44.286 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:35:44.286 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T11:35:44.286 INFO:tasks.ceph:Generating config... 2026-02-20T11:35:44.288 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-20T11:35:44.288 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-20T11:35:44.288 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-20T11:35:44.289 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-20T11:35:44.289 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-20T11:35:44.289 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-20T11:35:44.289 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-20T11:35:44.289 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-20T11:35:44.289 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-20T11:35:44.289 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-20T11:35:44.289 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-20T11:35:44.289 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-20T11:35:44.289 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-20T11:35:44.289 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-20T11:35:44.289 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-20T11:35:44.290 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-20T11:35:44.291 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-20T11:35:44.291 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-20T11:35:44.291 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-20T11:35:44.291 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-20T11:35:44.291 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-20T11:35:44.291 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-20T11:35:44.291 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-20T11:35:44.291 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-20T11:35:44.291 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-20T11:35:44.291 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-20T11:35:44.291 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-20T11:35:44.291 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-20T11:35:44.292 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-20T11:35:44.292 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-20T11:35:44.292 INFO:tasks.ceph:Setting up mon.a... 2026-02-20T11:35:44.292 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-20T11:35:44.367 INFO:teuthology.orchestra.run.trial072.stdout:creating /etc/ceph/ceph.keyring 2026-02-20T11:35:44.370 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-20T11:35:44.454 DEBUG:teuthology.orchestra.run.trial072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T11:35:44.520 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.72')] 2026-02-20T11:35:44.521 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.72', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-20T11:35:44.522 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:44.522 DEBUG:teuthology.orchestra.run.trial072:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T11:35:44.579 DEBUG:teuthology.orchestra.run.trial072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.72 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:35:44.659 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:35:44.657+0000 7f6573c98f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:35:44.660 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:35:44.658+0000 7f6573c98f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:35:44.660 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:35:44.658+0000 7f6573c98f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:35:44.661 INFO:teuthology.orchestra.run.trial072.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:35:44.661 INFO:teuthology.orchestra.run.trial072.stdout:monmaptool: generated fsid 52a34ca1-c1ae-495c-b1fd-496cfd636edc 2026-02-20T11:35:44.661 INFO:teuthology.orchestra.run.trial072.stdout:setting min_mon_release = tentacle 2026-02-20T11:35:44.661 INFO:teuthology.orchestra.run.trial072.stdout:epoch 0 2026-02-20T11:35:44.661 INFO:teuthology.orchestra.run.trial072.stdout:fsid 52a34ca1-c1ae-495c-b1fd-496cfd636edc 2026-02-20T11:35:44.662 INFO:teuthology.orchestra.run.trial072.stdout:last_changed 2026-02-20T11:35:44.659152+0000 2026-02-20T11:35:44.662 INFO:teuthology.orchestra.run.trial072.stdout:created 2026-02-20T11:35:44.659152+0000 2026-02-20T11:35:44.662 INFO:teuthology.orchestra.run.trial072.stdout:min_mon_release 20 (tentacle) 2026-02-20T11:35:44.662 INFO:teuthology.orchestra.run.trial072.stdout:election_strategy: 1 2026-02-20T11:35:44.662 INFO:teuthology.orchestra.run.trial072.stdout:0: [v2:10.20.193.72:3300/0,v1:10.20.193.72:6789/0] mon.a 2026-02-20T11:35:44.662 INFO:teuthology.orchestra.run.trial072.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-20T11:35:44.663 DEBUG:teuthology.orchestra.run.trial072:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T11:35:44.720 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 52a34ca1-c1ae-495c-b1fd-496cfd636edc... 2026-02-20T11:35:44.721 DEBUG:teuthology.orchestra.run.trial072:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-20T11:35:44.800 INFO:teuthology.orchestra.run.trial072.stdout:[global] 2026-02-20T11:35:44.800 INFO:teuthology.orchestra.run.trial072.stdout: chdir = "" 2026-02-20T11:35:44.800 INFO:teuthology.orchestra.run.trial072.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-20T11:35:44.800 INFO:teuthology.orchestra.run.trial072.stdout: auth supported = cephx 2026-02-20T11:35:44.800 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.800 INFO:teuthology.orchestra.run.trial072.stdout: filestore xattr use omap = true 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: mon clock drift allowed = 1.000 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: osd crush chooseleaf type = 0 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: auth debug = true 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: ms die on old message = true 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: ms die on bug = true 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: mon pg warn max object skew = 0 2026-02-20T11:35:44.801 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: # disable pg_autoscaler by default for new pools 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: osd pool default size = 2 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: mon osd allow primary affinity = true 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: mon osd allow pg remap = true 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: mon warn on legacy crush tunables = false 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: mon warn on crush straw calc version zero = false 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: mon warn on no sortbitwise = false 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: mon warn on osd down out interval zero = false 2026-02-20T11:35:44.802 INFO:teuthology.orchestra.run.trial072.stdout: mon warn on too few osds = false 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: mon_allow_pool_size_one = true 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: osd default data pool replay window = 5 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: mon allow pool delete = true 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: mon cluster log file level = debug 2026-02-20T11:35:44.803 INFO:teuthology.orchestra.run.trial072.stdout: debug asserts on shutdown = true 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: mon health detail to clog = false 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: mon host = 10.20.193.72 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: ms client mode = crc 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: ms cluster mode = crc 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: ms mon client mode = crc 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: ms mon cluster mode = crc 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: ms mon service mode = crc 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: ms service mode = crc 2026-02-20T11:35:44.804 INFO:teuthology.orchestra.run.trial072.stdout: fsid = 52a34ca1-c1ae-495c-b1fd-496cfd636edc 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout:[osd] 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: osd journal size = 100 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: osd scrub load threshold = 5.0 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: osd scrub max interval = 600 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: osd mclock profile = high_recovery_ops 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: osd mclock skip benchmark = true 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: osd recover clone overlap = true 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: osd recovery max chunk = 1048576 2026-02-20T11:35:44.805 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: osd debug shutdown = true 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: osd debug op order = true 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: osd debug verify stray on activate = true 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: osd debug trim objects = true 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: osd open classes on start = true 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: osd debug pg log writeout = true 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: osd deep scrub update digest min age = 30 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: osd map max advance = 10 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.806 INFO:teuthology.orchestra.run.trial072.stdout: journal zero on create = true 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: filestore ondisk finisher threads = 3 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: filestore apply finisher threads = 3 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: bdev debug aio = true 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: osd debug misdirected ops = true 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: debug monc = 20 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: crimson cpu num = 4 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: crimson osd obc lru size = 10 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: debug ms = 20 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: debug osd = 20 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore = 20 2026-02-20T11:35:44.807 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore backref = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore cache = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore cleaner = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore device = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore epm = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore fixedkv tree = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore journal = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore lba = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore odata = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore omap = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore onode = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore t = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: debug seastore tm = 20 2026-02-20T11:35:44.808 INFO:teuthology.orchestra.run.trial072.stdout: osd objectstore = seastore 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: osd shutdown pgref assert = True 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: seastore cachepin size pershard = 64M 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: seastore cachepin type = 2Q 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: seastore max concurrent transactions = 8 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout:[mgr] 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: debug ms = 1 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: debug mgr = 20 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: debug mon = 20 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: debug auth = 20 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: mon reweight min pgs per osd = 4 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: mon reweight min bytes per osd = 10 2026-02-20T11:35:44.809 INFO:teuthology.orchestra.run.trial072.stdout: mgr/telemetry/nag = false 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout:[mon] 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: debug ms = 1 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: debug mon = 20 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: debug paxos = 20 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: debug auth = 20 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: mon data avail warn = 5 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: mon mgr mkfs grace = 240 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: mon reweight min pgs per osd = 4 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: mon osd reporter subtree level = osd 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: mon osd prime pg temp = true 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: mon reweight min bytes per osd = 10 2026-02-20T11:35:44.810 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: auth mon ticket ttl = 660 # 11m 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: auth service ticket ttl = 240 # 4m 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: # don't complain about insecure global_id in the test suite 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: # 1m isn't quite enough 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: mon_down_mkfs_grace = 2m 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.811 INFO:teuthology.orchestra.run.trial072.stdout: mon_warn_on_filestore_osds = false 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: mon min osdmap epochs = 50 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: mon osdmap full prune interval = 2 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: mon osdmap full prune min = 15 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: mon osdmap full prune txsize = 2 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: paxos service trim min = 10 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: osd pool default crimson = True 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout:[client] 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: rgw cache enabled = true 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: rgw enable ops log = true 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: rgw enable usage log = true 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-20T11:35:44.812 INFO:teuthology.orchestra.run.trial072.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-20T11:35:44.813 INFO:teuthology.orchestra.run.trial072.stdout:[mon.a] 2026-02-20T11:35:44.814 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-20T11:35:44.814 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-20T11:35:44.906 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-20T11:35:44.906 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:44.906 DEBUG:teuthology.orchestra.run.trial072:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-20T11:35:44.923 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:44.923 DEBUG:teuthology.orchestra.run.trial072:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-20T11:35:44.980 INFO:tasks.ceph:Sending monmap to node ubuntu@trial072.front.sepia.ceph.com 2026-02-20T11:35:44.980 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:44.980 DEBUG:teuthology.orchestra.run.trial072:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-20T11:35:44.980 DEBUG:teuthology.orchestra.run.trial072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T11:35:45.054 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:35:45.054 DEBUG:teuthology.orchestra.run.trial072:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:35:45.109 INFO:tasks.ceph:Setting up mon nodes... 2026-02-20T11:35:45.110 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-20T11:35:45.110 DEBUG:teuthology.orchestra.run.trial072:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T11:35:45.200 INFO:teuthology.orchestra.run.trial072.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T11:35:45.203 INFO:tasks.ceph:Setting up mds nodes... 2026-02-20T11:35:45.203 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-20T11:35:45.203 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-20T11:35:45.245 INFO:teuthology.orchestra.run.trial072.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-20T11:35:45.256 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-20T11:35:45.257 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T11:35:45.257 DEBUG:teuthology.orchestra.run.trial072:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-20T11:35:45.321 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:35:45.322 INFO:tasks.ceph:role: osd.0 2026-02-20T11:35:45.322 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial072.front.sepia.ceph.com 2026-02-20T11:35:45.322 DEBUG:teuthology.orchestra.run.trial072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-20T11:35:45.388 INFO:teuthology.orchestra.run.trial072.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:35:45.388 INFO:teuthology.orchestra.run.trial072.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:35:45.388 INFO:teuthology.orchestra.run.trial072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:35:45.388 INFO:teuthology.orchestra.run.trial072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:35:45.388 INFO:teuthology.orchestra.run.trial072.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:35:45.388 INFO:teuthology.orchestra.run.trial072.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:35:45.389 INFO:teuthology.orchestra.run.trial072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:35:45.389 INFO:teuthology.orchestra.run.trial072.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:35:45.389 INFO:teuthology.orchestra.run.trial072.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:35:45.389 INFO:teuthology.orchestra.run.trial072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:35:45.958 INFO:teuthology.orchestra.run.trial072.stdout:Discarding blocks...Done. 2026-02-20T11:35:45.960 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial072.front.sepia.ceph.com -o noatime 2026-02-20T11:35:45.960 DEBUG:teuthology.orchestra.run.trial072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-20T11:35:46.029 DEBUG:teuthology.orchestra.run.trial072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-20T11:35:46.098 DEBUG:teuthology.orchestra.run.trial072:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-20T11:35:46.162 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:35:46.162 INFO:tasks.ceph:role: osd.1 2026-02-20T11:35:46.162 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial072.front.sepia.ceph.com 2026-02-20T11:35:46.162 DEBUG:teuthology.orchestra.run.trial072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:35:46.229 INFO:teuthology.orchestra.run.trial072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:35:46.786 INFO:teuthology.orchestra.run.trial072.stdout:Discarding blocks...Done. 2026-02-20T11:35:46.788 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial072.front.sepia.ceph.com -o noatime 2026-02-20T11:35:46.788 DEBUG:teuthology.orchestra.run.trial072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-20T11:35:46.857 DEBUG:teuthology.orchestra.run.trial072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-20T11:35:46.925 DEBUG:teuthology.orchestra.run.trial072:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-20T11:35:46.989 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:35:46.989 INFO:tasks.ceph:role: osd.2 2026-02-20T11:35:46.989 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial072.front.sepia.ceph.com 2026-02-20T11:35:46.989 DEBUG:teuthology.orchestra.run.trial072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-20T11:35:47.054 INFO:teuthology.orchestra.run.trial072.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:35:47.054 INFO:teuthology.orchestra.run.trial072.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:35:47.054 INFO:teuthology.orchestra.run.trial072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:35:47.054 INFO:teuthology.orchestra.run.trial072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:35:47.054 INFO:teuthology.orchestra.run.trial072.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:35:47.055 INFO:teuthology.orchestra.run.trial072.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:35:47.055 INFO:teuthology.orchestra.run.trial072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:35:47.055 INFO:teuthology.orchestra.run.trial072.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:35:47.055 INFO:teuthology.orchestra.run.trial072.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:35:47.055 INFO:teuthology.orchestra.run.trial072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:35:47.611 INFO:teuthology.orchestra.run.trial072.stdout:Discarding blocks...Done. 2026-02-20T11:35:47.613 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial072.front.sepia.ceph.com -o noatime 2026-02-20T11:35:47.613 DEBUG:teuthology.orchestra.run.trial072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-20T11:35:47.679 DEBUG:teuthology.orchestra.run.trial072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-20T11:35:47.746 DEBUG:teuthology.orchestra.run.trial072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:35:47.898 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:35:47.956 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:47,955 seastar - Reactor backend: linux-aio 2026-02-20T11:35:47.969 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:47,968 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:47.969 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:47,968 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:47.971 INFO:teuthology.orchestra.run.trial072.stderr:==39232==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:35:48.027 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:48,026 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:35:48.027 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:48,026 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:35:48.028 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:48,026 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:35:48.028 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:48,026 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:35:48.028 INFO:teuthology.orchestra.run.trial072.stderr:DEBUG 2026-02-20 11:35:48,026 [shard 0:main] seastar - reactor::drain 2026-02-20T11:35:50.087 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:35:50.087 INFO:teuthology.orchestra.run.trial072.stderr:Suppressions used: 2026-02-20T11:35:50.087 INFO:teuthology.orchestra.run.trial072.stderr: count bytes template 2026-02-20T11:35:50.087 INFO:teuthology.orchestra.run.trial072.stderr: 1 8 MallocExtension::Register 2026-02-20T11:35:50.087 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:35:50.088 INFO:teuthology.orchestra.run.trial072.stderr: 2026-02-20T11:35:50.106 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:35:50.107 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:35:50.151 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:35:50.151 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:35:50.157 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:50,157 seastar - Reactor backend: linux-aio 2026-02-20T11:35:50.168 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:50,168 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:50.168 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:50,168 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:50.172 INFO:teuthology.orchestra.run.trial072.stderr:==39230==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:35:50.172 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:50,172 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:35:50.229 INFO:teuthology.orchestra.run.trial072.stderr:DEBUG 2026-02-20 11:35:50,228 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:35:50.310 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcaf3 /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15e37 0x9e1e650 0x9e1ea02 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.310 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e2cc92 0x9efa76f 0x9efa937 0x9efaa62 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.310 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc657f /lib64/libasan.so.8+0x16bd7 0x9f0e754 0x9f13918 0x9f13f56 0x9f1e84c 0x78bcf19 0x7b9c081 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.343 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16f348 /lib64/libasan.so.8+0x4d695 0x30b4ba3 0x30f33b6 0x4cb11cb 0x9efa8f3 0x9efaa62 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b8bc1a 0x7ba0607 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:35:50.386 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65d8 /lib64/libasan.so.8+0x157c7 0x10a3723 0xf4865a 0x9e22cec 0x9e22f99 0x9f17991 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7e49eb9 0x7e534a3 0x79aa9cb 0x7a40cd7 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.386 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x30e87bf 0x30ea30d 0x344d651 0x333c3ad 0x33609c9 0x3361616 0x33d0a09 0x33d2511 0x33d3316 0xd14917 0xd14bca 0xe61245 0x90ebcca 0x90ec1c0 0x9167135 0x7a42cf7 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.386 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191cf5 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x1367182 0x29df4ae 0x7f6c018 0x7f6c5ce 0x7f6c6fa 0x7f6f26f 0x7f6fec3 0x7f70184 0x7e5a8a9 0x7fb3f68 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.386 INFO:teuthology.orchestra.run.trial072.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffbbd5fe1b 0xffffffffbb38ab94 0xffffffffbb38ac8b 0xffffffffbb3cb948 0xffffffffbb3d3aeb 0xffffffffbb3d3f4b 0xffffffffbb0910d2 0xffffffffbbd4d451 0xffffffffbbe00ba2 2026-02-20T11:35:50.411 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d88f6 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:35:50.453 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6583 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.453 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.454 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f494 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.581 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x9e1e32e 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.581 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x10a37c7 0xf4865a 0x9e22cec 0x9e2362b 0x9f16b01 0x9f17a21 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.581 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b364 /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xb7b689 0x10a36ac 0xf4865a 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.664 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129747 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:35:50.681 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f494 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.681 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ed7 /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:50.681 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0f /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:52.878 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:35:52.878 INFO:teuthology.orchestra.run.trial072.stderr:Suppressions used: 2026-02-20T11:35:52.878 INFO:teuthology.orchestra.run.trial072.stderr: count bytes template 2026-02-20T11:35:52.878 INFO:teuthology.orchestra.run.trial072.stderr: 1 8 MallocExtension::Register 2026-02-20T11:35:52.878 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:35:52.878 INFO:teuthology.orchestra.run.trial072.stderr: 2026-02-20T11:35:52.913 DEBUG:teuthology.orchestra.run.trial072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-20T11:35:52.938 DEBUG:teuthology.orchestra.run.trial072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:35:53.092 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:35:53.150 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:53,149 seastar - Reactor backend: linux-aio 2026-02-20T11:35:53.159 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:53,158 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:53.159 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:53,158 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:53.161 INFO:teuthology.orchestra.run.trial072.stderr:==39300==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:35:53.217 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:53,216 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:35:53.218 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:53,216 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:35:53.218 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:53,216 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:35:53.218 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:53,216 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:35:53.218 INFO:teuthology.orchestra.run.trial072.stderr:DEBUG 2026-02-20 11:35:53,216 [shard 0:main] seastar - reactor::drain 2026-02-20T11:35:55.270 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:35:55.270 INFO:teuthology.orchestra.run.trial072.stderr:Suppressions used: 2026-02-20T11:35:55.270 INFO:teuthology.orchestra.run.trial072.stderr: count bytes template 2026-02-20T11:35:55.270 INFO:teuthology.orchestra.run.trial072.stderr: 1 8 MallocExtension::Register 2026-02-20T11:35:55.270 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:35:55.270 INFO:teuthology.orchestra.run.trial072.stderr: 2026-02-20T11:35:55.290 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:35:55.290 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:35:55.331 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:35:55.331 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:35:55.338 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:55,337 seastar - Reactor backend: linux-aio 2026-02-20T11:35:55.349 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:55,348 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:55.350 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:55,348 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:55.354 INFO:teuthology.orchestra.run.trial072.stderr:==39298==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:35:55.354 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:55,353 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:35:55.412 INFO:teuthology.orchestra.run.trial072.stderr:DEBUG 2026-02-20 11:35:55,411 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:35:55.494 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151dd 0x1138fc8 0xf4864f 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.494 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xccaa98 0x10a369e 0xf4865a 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e20729 0x9f0db89 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.494 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe419a1 0x9edc84a 0x9edd05c 0x9eebabe 0x9eee549 0x9eee75a 0x9eee806 0x9f17355 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.494 INFO:teuthology.orchestra.run.trial072.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffbb075c69 0xffffffffbb21f8c7 0xffffffffbb21fb20 0xffffffffbb09525e 0xffffffffbb3e61a9 0xffffffffbb3c9715 0xffffffffbb3d3aeb 0xffffffffbb3d3f4b 0xffffffffbb0910d2 0xffffffffbbd4d451 0xffffffffbbe00ba2 2026-02-20T11:35:55.526 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xea81 /lib64/libasan.so.8+0xef905 0x320034c 0x3200526 0x32007dd 0x32007a2 0x32008d2 0x9f1aa95 0x9edc6d1 0x9edb6d1 0x7b8bc1a 0x7ba0607 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:35:55.570 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x1367182 0x29df4ae 0x7f6c018 0x7f6c5ce 0x7f6c6fa 0x7f6f26f 0x7f6fec3 0x7f70184 0x7e5a8a9 0x7fb3f68 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.570 INFO:teuthology.orchestra.run.trial072.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffbb21f6b4 0xffffffffbb21fb20 0xffffffffbb09525e 0xffffffffbb3e61a9 0xffffffffbb3c9715 0xffffffffbb3d3aeb 0xffffffffbb3d3f4b 0xffffffffbb0910d2 0xffffffffbbd4d451 0xffffffffbbe00ba2 2026-02-20T11:35:55.570 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ed3 /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x157c7 0x7f6b1b3 0x7f6c00d 0x7f6c5ce 0x7f6c6fa 0x7f6f26f 0x7f6fec3 0x7f70184 0x7e5a8a9 0x7fb3f68 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.571 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xeb96 /lib64/libasan.so.8+0xef905 0x7d8a1c1 0x7beace1 0x7beaf03 0x7beaec8 0x7c05164 0x7ba28d3 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.594 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x341f /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:35:55.636 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.636 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d89b8 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.637 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d88f6 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.761 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e2cc52 0x9efa76f 0x9efa937 0x9efaa98 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.761 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15835 0x3549a64 0x3549be4 0x359d2e4 0x9e30705 0x9e307e8 0x9ef7d44 0x9ef7ed9 0x9ef826b 0x9ef9851 0x9efab6c 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.761 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1586f 0x10a3723 0xf4865a 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e20729 0x9f0b606 0x9f0d258 0x9f0d8b2 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.843 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d7d76 0x7d69b1d 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:35:55.859 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f496 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.859 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:55.860 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f494 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:35:58.063 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:35:58.063 INFO:teuthology.orchestra.run.trial072.stderr:Suppressions used: 2026-02-20T11:35:58.063 INFO:teuthology.orchestra.run.trial072.stderr: count bytes template 2026-02-20T11:35:58.063 INFO:teuthology.orchestra.run.trial072.stderr: 1 8 MallocExtension::Register 2026-02-20T11:35:58.063 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:35:58.064 INFO:teuthology.orchestra.run.trial072.stderr: 2026-02-20T11:35:58.099 DEBUG:teuthology.orchestra.run.trial072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-20T11:35:58.125 DEBUG:teuthology.orchestra.run.trial072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:35:58.279 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:35:58.335 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:58,334 seastar - Reactor backend: linux-aio 2026-02-20T11:35:58.345 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:58,345 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:58.346 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:58,345 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:35:58.348 INFO:teuthology.orchestra.run.trial072.stderr:==39367==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:35:58.403 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:58,402 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T11:35:58.403 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:58,402 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T11:35:58.403 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:58,402 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T11:35:58.403 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:35:58,402 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T11:35:58.403 INFO:teuthology.orchestra.run.trial072.stderr:DEBUG 2026-02-20 11:35:58,403 [shard 0:main] seastar - reactor::drain 2026-02-20T11:36:00.488 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:36:00.488 INFO:teuthology.orchestra.run.trial072.stderr:Suppressions used: 2026-02-20T11:36:00.488 INFO:teuthology.orchestra.run.trial072.stderr: count bytes template 2026-02-20T11:36:00.488 INFO:teuthology.orchestra.run.trial072.stderr: 1 8 MallocExtension::Register 2026-02-20T11:36:00.489 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:36:00.489 INFO:teuthology.orchestra.run.trial072.stderr: 2026-02-20T11:36:00.508 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:36:00.508 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:36:00.553 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:36:00.553 INFO:teuthology.orchestra.run.trial072.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:36:00.560 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:36:00,559 seastar - Reactor backend: linux-aio 2026-02-20T11:36:00.571 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:36:00,570 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:36:00.571 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:36:00,570 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:36:00.575 INFO:teuthology.orchestra.run.trial072.stderr:==39365==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:36:00.575 INFO:teuthology.orchestra.run.trial072.stderr:INFO 2026-02-20 11:36:00,574 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:36:00.633 INFO:teuthology.orchestra.run.trial072.stderr:DEBUG 2026-02-20 11:36:00,632 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T11:36:00.714 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1586f 0x10a3723 0xf4865a 0x9f085d5 0x9f08bc2 0x9f08f45 0x9f1909c 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.714 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65d8 /lib64/libasan.so.8+0x15197 0x1138fc8 0xf4864f 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.716 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 34 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xcc324f 0x1138f42 0xf4864f 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.746 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x10a3666 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:36:00.789 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x7ae3584 0x795e987 0x795ef0f 0x790698a 0x794203b 0x7a3f2be 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.789 INFO:teuthology.orchestra.run.trial072.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffbbd5fe1b 0xffffffffbb38ab94 0xffffffffbb38ac8b 0xffffffffbb3c972f 0xffffffffbb3d3aeb 0xffffffffbb3d3f4b 0xffffffffbb0910d2 0xffffffffbbd4d451 0xffffffffbbe00ba2 2026-02-20T11:36:00.790 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xb86d77 0xb872f9 0xb874d8 0xb87555 0xc7ed6b 0x960a52b 0x960a905 0x960aa64 0xa81afb8 0xa81b5aa 0x95d1578 0x96243fb 0x90ebcea 0x90ec1c0 0x91669ba 0x7a42cf7 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.792 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe419a1 0x9edc84a 0x9f19457 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x911c2c7 0x911d4dc 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.792 INFO:teuthology.orchestra.run.trial072.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffbb0912ca 0xffffffffbbd4d451 0xffffffffbbe00ba2 2026-02-20T11:36:00.817 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d7a18 0x7d69b08 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:36:00.826 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d89e3 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.856 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d79b5 0x7d69b08 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.857 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.988 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e2cc52 0x9efa76f 0x9efa937 0x9efaa98 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.988 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x31f9adf 0x9e1e750 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:00.989 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e1ebd5 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:01.071 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d83b5 0x7d69b2a 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:36:01.086 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f494 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:01.087 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f46e 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:01.087 INFO:teuthology.orchestra.run.trial072.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d83fc 0x7d69b2a 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:36:03.270 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:36:03.270 INFO:teuthology.orchestra.run.trial072.stderr:Suppressions used: 2026-02-20T11:36:03.270 INFO:teuthology.orchestra.run.trial072.stderr: count bytes template 2026-02-20T11:36:03.270 INFO:teuthology.orchestra.run.trial072.stderr: 1 8 MallocExtension::Register 2026-02-20T11:36:03.270 INFO:teuthology.orchestra.run.trial072.stderr:----------------------------------------------------- 2026-02-20T11:36:03.270 INFO:teuthology.orchestra.run.trial072.stderr: 2026-02-20T11:36:03.305 DEBUG:teuthology.orchestra.run.trial072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-20T11:36:03.330 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-20T11:36:03.330 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:03.330 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-20T11:36:03.394 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:03.394 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-20T11:36:03.459 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:03.459 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-20T11:36:03.523 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:03.523 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-20T11:36:03.588 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:03.588 DEBUG:teuthology.orchestra.run.trial072:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-20T11:36:03.645 INFO:tasks.ceph:Adding keys to all mons... 2026-02-20T11:36:03.645 DEBUG:teuthology.orchestra.run.trial072:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-20T11:36:03.707 INFO:teuthology.orchestra.run.trial072.stdout:[mgr.x] 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout: key = AQARR5hp7mzoCxAA9THVjooePcJS9UCPyi4VOg== 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout:[osd.0] 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout: key = AQAWR5hpP1TfDRAARFSphRD+5h8+14N4JWQrBA== 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout:[osd.1] 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout: key = AQAbR5hpzrzFGBAAIN/KaC/AOxr3H92XwM1laA== 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout:[osd.2] 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout: key = AQAgR5hpfBnuJRAAg/kpgFsazr1+4ug4FhpDPg== 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout:[client.0] 2026-02-20T11:36:03.708 INFO:teuthology.orchestra.run.trial072.stdout: key = AQARR5hpbV2TDhAAWe6XutHLz2FAesb3Tf8tDQ== 2026-02-20T11:36:03.710 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-20T11:36:03.798 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:36:03.847 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:36:03.894 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:36:03.941 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-20T11:36:03.990 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-20T11:36:03.990 DEBUG:teuthology.orchestra.run.trial072:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-20T11:36:04.014 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-20T11:36:04.100 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:04.099+0000 7fe4d0d23c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.100 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:04.099+0000 7fe4d0d23c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.111 DEBUG:teuthology.orchestra.run.trial072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-20T11:36:04.137 DEBUG:teuthology.orchestra.run.trial072:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:36:04.193 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-20T11:36:04.194 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-20T11:36:04.194 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-20T11:36:04.236 INFO:tasks.ceph.mon.a:Started 2026-02-20T11:36:04.236 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-20T11:36:04.236 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-20T11:36:04.236 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-20T11:36:04.239 INFO:tasks.ceph.mgr.x:Started 2026-02-20T11:36:04.239 DEBUG:tasks.ceph:set 0 configs 2026-02-20T11:36:04.239 DEBUG:teuthology.orchestra.run.trial072:> sudo ceph --cluster ceph config dump 2026-02-20T11:36:04.301 INFO:tasks.ceph.mon.a.trial072.stderr:2026-02-20T11:36:04.299+0000 7f8a478b6c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.302 INFO:tasks.ceph.mon.a.trial072.stderr:2026-02-20T11:36:04.300+0000 7f8a478b6c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.302 INFO:tasks.ceph.mon.a.trial072.stderr:2026-02-20T11:36:04.300+0000 7f8a478b6c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.303 INFO:tasks.ceph.mgr.x.trial072.stderr:2026-02-20T11:36:04.302+0000 7f0b0bdc4fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.316 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:04.313+0000 7f8183b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.319 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:04.318+0000 7f8183b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.352 INFO:teuthology.orchestra.run.trial072.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-20T11:36:04.361 INFO:tasks.ceph:Setting crush tunables to default 2026-02-20T11:36:04.362 DEBUG:teuthology.orchestra.run.trial072:> sudo ceph --cluster ceph osd crush tunables default 2026-02-20T11:36:04.435 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:04.430+0000 7f5f1f2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.437 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:04.436+0000 7f5f1f2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.472 INFO:teuthology.orchestra.run.trial072.stderr:adjusted tunables profile to default 2026-02-20T11:36:04.480 INFO:tasks.ceph:check_enable_crimson: True 2026-02-20T11:36:04.481 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-20T11:36:04.481 DEBUG:teuthology.orchestra.run.trial072:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-20T11:36:04.507 INFO:tasks.ceph.mgr.x.trial072.stderr:2026-02-20T11:36:04.506+0000 7f0b0bdc4fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.508 INFO:tasks.ceph.mgr.x.trial072.stderr:2026-02-20T11:36:04.506+0000 7f0b0bdc4fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.556 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:04.552+0000 7f586d806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.559 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:04.557+0000 7f586d806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:04.598 INFO:tasks.ceph.mon.a.trial072.stderr:2026-02-20T11:36:04.597+0000 7f8a3eff1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-20T11:36:04.598 INFO:tasks.ceph.mon.a.trial072.stderr:Please be aware that this feature is experimental, untested, 2026-02-20T11:36:04.598 INFO:tasks.ceph.mon.a.trial072.stderr:unsupported, and may result in data corruption, data loss, 2026-02-20T11:36:04.598 INFO:tasks.ceph.mon.a.trial072.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-20T11:36:04.599 INFO:tasks.ceph.mon.a.trial072.stderr:feature with important data. 2026-02-20T11:36:04.599 INFO:tasks.ceph.mon.a.trial072.stderr: 2026-02-20T11:36:05.348 INFO:tasks.ceph.mgr.x.trial072.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-20T11:36:05.348 INFO:tasks.ceph.mgr.x.trial072.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-20T11:36:05.348 INFO:tasks.ceph.mgr.x.trial072.stderr: from numpy import show_config as show_numpy_config 2026-02-20T11:36:05.482 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-20T11:36:05.483 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:05.483 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-20T11:36:05.547 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:05.547 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-20T11:36:05.611 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:05.611 DEBUG:teuthology.orchestra.run.trial072:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-20T11:36:05.675 DEBUG:teuthology.orchestra.run.trial072:> sudo ceph --cluster ceph osd new d8a79b48-9131-49f1-a45a-18fe5a376bb7 0 2026-02-20T11:36:05.789 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:05.784+0000 7f055110b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:05.791 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:05.790+0000 7f055110b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:05.829 INFO:teuthology.orchestra.run.trial072.stdout:0 2026-02-20T11:36:05.837 DEBUG:teuthology.orchestra.run.trial072:> sudo ceph --cluster ceph osd new f8b45f80-8e99-424d-bbcc-29a3a63e3e53 1 2026-02-20T11:36:05.911 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:05.906+0000 7fcd2a7b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:05.913 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:05.911+0000 7fcd2a7b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:05.951 INFO:teuthology.orchestra.run.trial072.stdout:1 2026-02-20T11:36:05.960 DEBUG:teuthology.orchestra.run.trial072:> sudo ceph --cluster ceph osd new 84a38c7f-8900-4f88-947e-b9bbe2c28399 2 2026-02-20T11:36:06.036 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:06.031+0000 7f82345b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:06.038 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:06.036+0000 7f82345b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:06.076 INFO:teuthology.orchestra.run.trial072.stdout:2 2026-02-20T11:36:06.084 DEBUG:teuthology.orchestra.run.trial072:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-20T11:36:06.112 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-20T11:36:06.113 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-20T11:36:06.154 INFO:tasks.ceph.osd.0:Started 2026-02-20T11:36:06.155 DEBUG:teuthology.orchestra.run.trial072:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-20T11:36:06.183 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-20T11:36:06.184 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-20T11:36:06.226 INFO:tasks.ceph.osd.1:Started 2026-02-20T11:36:06.227 DEBUG:teuthology.orchestra.run.trial072:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-20T11:36:06.256 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-20T11:36:06.256 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-20T11:36:06.297 INFO:tasks.ceph.osd.2:Started 2026-02-20T11:36:06.298 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:36:06.377 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:06.373+0000 7f35e7b64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:06.380 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:06.378+0000 7f35e7b64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:06.414 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:36:06.414 INFO:teuthology.orchestra.run.trial072.stdout:{"epoch":6,"fsid":"52a34ca1-c1ae-495c-b1fd-496cfd636edc","created":"2026-02-20T11:36:04.312001+0000","modified":"2026-02-20T11:36:06.074942+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:36:06.074942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"d8a79b48-9131-49f1-a45a-18fe5a376bb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f8b45f80-8e99-424d-bbcc-29a3a63e3e53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"84a38c7f-8900-4f88-947e-b9bbe2c28399","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:36:06.423 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-20T11:36:06.424 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-20T11:36:06.726 DEBUG:teuthology.orchestra.run.trial072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:36:06.798 INFO:teuthology.misc.health.trial072.stderr:2026-02-20T11:36:06.794+0000 7f1e223b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:06.800 INFO:teuthology.misc.health.trial072.stderr:2026-02-20T11:36:06.798+0000 7f1e223b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:06.832 INFO:teuthology.misc.health.trial072.stdout: 2026-02-20T11:36:06.832 INFO:teuthology.misc.health.trial072.stdout:{"epoch":6,"fsid":"52a34ca1-c1ae-495c-b1fd-496cfd636edc","created":"2026-02-20T11:36:04.312001+0000","modified":"2026-02-20T11:36:06.074942+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:36:06.074942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"d8a79b48-9131-49f1-a45a-18fe5a376bb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f8b45f80-8e99-424d-bbcc-29a3a63e3e53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"84a38c7f-8900-4f88-947e-b9bbe2c28399","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:36:06.839 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T11:36:07.771 INFO:tasks.ceph.mgr.x.trial072.stderr:2026-02-20T11:36:07.770+0000 7f0a26d03640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-20T11:36:13.142 DEBUG:teuthology.orchestra.run.trial072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:36:13.215 INFO:teuthology.misc.health.trial072.stderr:2026-02-20T11:36:13.212+0000 7fbb82a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:13.218 INFO:teuthology.misc.health.trial072.stderr:2026-02-20T11:36:13.216+0000 7fbb82a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:13.343 INFO:teuthology.misc.health.trial072.stdout: 2026-02-20T11:36:13.343 INFO:teuthology.misc.health.trial072.stdout:{"epoch":8,"fsid":"52a34ca1-c1ae-495c-b1fd-496cfd636edc","created":"2026-02-20T11:36:04.312001+0000","modified":"2026-02-20T11:36:12.648765+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:36:06.074942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"d8a79b48-9131-49f1-a45a-18fe5a376bb7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f8b45f80-8e99-424d-bbcc-29a3a63e3e53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"84a38c7f-8900-4f88-947e-b9bbe2c28399","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:36:13.356 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T11:36:19.658 DEBUG:teuthology.orchestra.run.trial072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:36:19.731 INFO:teuthology.misc.health.trial072.stderr:2026-02-20T11:36:19.726+0000 7f277b846640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:19.733 INFO:teuthology.misc.health.trial072.stderr:2026-02-20T11:36:19.731+0000 7f277b846640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:19.845 INFO:teuthology.misc.health.trial072.stdout: 2026-02-20T11:36:19.845 INFO:teuthology.misc.health.trial072.stdout:{"epoch":14,"fsid":"52a34ca1-c1ae-495c-b1fd-496cfd636edc","created":"2026-02-20T11:36:04.312001+0000","modified":"2026-02-20T11:36:19.676125+0000","last_up_change":"2026-02-20T11:36:15.664562+0000","last_in_change":"2026-02-20T11:36:06.074942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:36:16.784096+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d8a79b48-9131-49f1-a45a-18fe5a376bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6803","nonce":201038210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6802","nonce":201038210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6806","nonce":201038210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6807","nonce":201038210}]},"public_addr":"10.20.193.72:6803/201038210","cluster_addr":"10.20.193.72:6802/201038210","heartbeat_back_addr":"10.20.193.72:6806/201038210","heartbeat_front_addr":"10.20.193.72:6807/201038210","state":["exists","up"]},{"osd":1,"uuid":"f8b45f80-8e99-424d-bbcc-29a3a63e3e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6808","nonce":252753268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6809","nonce":252753268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6813","nonce":252753268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6814","nonce":252753268}]},"public_addr":"10.20.193.72:6808/252753268","cluster_addr":"10.20.193.72:6809/252753268","heartbeat_back_addr":"10.20.193.72:6813/252753268","heartbeat_front_addr":"10.20.193.72:6814/252753268","state":["exists","up"]},{"osd":2,"uuid":"84a38c7f-8900-4f88-947e-b9bbe2c28399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6805","nonce":2869470225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6804","nonce":2869470225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6812","nonce":2869470225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6811","nonce":2869470225}]},"public_addr":"10.20.193.72:6805/2869470225","cluster_addr":"10.20.193.72:6804/2869470225","heartbeat_back_addr":"10.20.193.72:6812/2869470225","heartbeat_front_addr":"10.20.193.72:6811/2869470225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:36:19.854 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-20T11:36:19.854 INFO:tasks.ceph:Creating RBD pool 2026-02-20T11:36:19.854 DEBUG:teuthology.orchestra.run.trial072:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-20T11:36:19.929 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:19.923+0000 7f88e377e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:19.931 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:19.929+0000 7f88e377e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:20.682 INFO:teuthology.orchestra.run.trial072.stderr:pool 'rbd' created 2026-02-20T11:36:20.695 DEBUG:teuthology.orchestra.run.trial072:> rbd --cluster ceph pool init rbd 2026-02-20T11:36:20.736 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:20.733+0000 7f81dc4df240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:20.736 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:20.733+0000 7f81dc4df240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:20.739 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:20.736+0000 7f81dc4df240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:23.728 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-20T11:36:23.729 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-20T11:36:23.730 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-20T11:36:23.846 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:23.841+0000 7f391c05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:23.848 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:23.846+0000 7f391c05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:23.959 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:36:23.969 INFO:teuthology.orchestra.run.trial072.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-20T11:36:23.969 INFO:tasks.ceph_manager:config epoch is 1 2026-02-20T11:36:23.969 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-20T11:36:23.969 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-20T11:36:23.970 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-20T11:36:24.048 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.043+0000 7f54d0993640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.050 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.048+0000 7f54d0993640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.192 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:36:24.201 INFO:teuthology.orchestra.run.trial072.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6800","nonce":2672921765},{"type":"v1","addr":"10.20.193.72:6801","nonce":2672921765}]},"active_addr":"10.20.193.72:6801/2672921765","active_change":"2026-02-20T11:36:06.755030+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.72:0","nonce":2664632009}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.72:0","nonce":1274426890}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.72:0","nonce":1725739541}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.72:0","nonce":3001789147}]}]} 2026-02-20T11:36:24.205 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-20T11:36:24.205 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-20T11:36:24.205 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:36:24.283 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.278+0000 7f8d90a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.285 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.282+0000 7f8d90a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.396 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:36:24.396 INFO:teuthology.orchestra.run.trial072.stdout:{"epoch":18,"fsid":"52a34ca1-c1ae-495c-b1fd-496cfd636edc","created":"2026-02-20T11:36:04.312001+0000","modified":"2026-02-20T11:36:23.700153+0000","last_up_change":"2026-02-20T11:36:15.664562+0000","last_in_change":"2026-02-20T11:36:06.074942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:36:16.784096+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:36:20.046554+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d8a79b48-9131-49f1-a45a-18fe5a376bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6803","nonce":201038210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6802","nonce":201038210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6806","nonce":201038210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6807","nonce":201038210}]},"public_addr":"10.20.193.72:6803/201038210","cluster_addr":"10.20.193.72:6802/201038210","heartbeat_back_addr":"10.20.193.72:6806/201038210","heartbeat_front_addr":"10.20.193.72:6807/201038210","state":["exists","up"]},{"osd":1,"uuid":"f8b45f80-8e99-424d-bbcc-29a3a63e3e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6808","nonce":252753268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6809","nonce":252753268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6813","nonce":252753268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6814","nonce":252753268}]},"public_addr":"10.20.193.72:6808/252753268","cluster_addr":"10.20.193.72:6809/252753268","heartbeat_back_addr":"10.20.193.72:6813/252753268","heartbeat_front_addr":"10.20.193.72:6814/252753268","state":["exists","up"]},{"osd":2,"uuid":"84a38c7f-8900-4f88-947e-b9bbe2c28399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6805","nonce":2869470225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6804","nonce":2869470225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6812","nonce":2869470225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6811","nonce":2869470225}]},"public_addr":"10.20.193.72:6805/2869470225","cluster_addr":"10.20.193.72:6804/2869470225","heartbeat_back_addr":"10.20.193.72:6812/2869470225","heartbeat_front_addr":"10.20.193.72:6811/2869470225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:36:24.406 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-20T11:36:24.406 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:36:24.483 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.479+0000 7efc70374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.485 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.483+0000 7efc70374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.596 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:36:24.596 INFO:teuthology.orchestra.run.trial072.stdout:{"epoch":18,"fsid":"52a34ca1-c1ae-495c-b1fd-496cfd636edc","created":"2026-02-20T11:36:04.312001+0000","modified":"2026-02-20T11:36:23.700153+0000","last_up_change":"2026-02-20T11:36:15.664562+0000","last_in_change":"2026-02-20T11:36:06.074942+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:36:16.784096+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:36:20.046554+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d8a79b48-9131-49f1-a45a-18fe5a376bb7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6803","nonce":201038210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6802","nonce":201038210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6806","nonce":201038210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6807","nonce":201038210}]},"public_addr":"10.20.193.72:6803/201038210","cluster_addr":"10.20.193.72:6802/201038210","heartbeat_back_addr":"10.20.193.72:6806/201038210","heartbeat_front_addr":"10.20.193.72:6807/201038210","state":["exists","up"]},{"osd":1,"uuid":"f8b45f80-8e99-424d-bbcc-29a3a63e3e53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6808","nonce":252753268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6809","nonce":252753268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6813","nonce":252753268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6814","nonce":252753268}]},"public_addr":"10.20.193.72:6808/252753268","cluster_addr":"10.20.193.72:6809/252753268","heartbeat_back_addr":"10.20.193.72:6813/252753268","heartbeat_front_addr":"10.20.193.72:6814/252753268","state":["exists","up"]},{"osd":2,"uuid":"84a38c7f-8900-4f88-947e-b9bbe2c28399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6805","nonce":2869470225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6804","nonce":2869470225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6812","nonce":2869470225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.72:6811","nonce":2869470225}]},"public_addr":"10.20.193.72:6805/2869470225","cluster_addr":"10.20.193.72:6804/2869470225","heartbeat_back_addr":"10.20.193.72:6812/2869470225","heartbeat_front_addr":"10.20.193.72:6811/2869470225","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:36:24.605 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-20T11:36:24.605 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-20T11:36:24.605 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-20T11:36:24.687 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.682+0000 7fc59b2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.688 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.683+0000 7f47cfdf5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.688 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.683+0000 7f798e68b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.689 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.688+0000 7fc59b2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.690 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.688+0000 7f47cfdf5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.690 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.688+0000 7f798e68b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.708 INFO:teuthology.orchestra.run.trial072.stdout:47244640265 2026-02-20T11:36:24.708 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:36:24.709 INFO:teuthology.orchestra.run.trial072.stdout:47244640265 2026-02-20T11:36:24.709 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:36:24.709 INFO:teuthology.orchestra.run.trial072.stdout:38654705675 2026-02-20T11:36:24.709 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T11:36:24.830 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.824+0000 7f456268b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.831 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.825+0000 7f3a40f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.831 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.825+0000 7f96cac1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.832 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.830+0000 7f456268b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.833 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.831+0000 7f3a40f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.833 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:24.831+0000 7f96cac1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:24.950 INFO:teuthology.orchestra.run.trial072.stdout:47244640261 2026-02-20T11:36:24.953 INFO:teuthology.orchestra.run.trial072.stdout:38654705669 2026-02-20T11:36:24.955 INFO:teuthology.orchestra.run.trial072.stdout:47244640261 2026-02-20T11:36:24.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640265 got 47244640261 for osd.2 2026-02-20T11:36:24.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705675 got 38654705669 for osd.0 2026-02-20T11:36:24.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640265 got 47244640261 for osd.1 2026-02-20T11:36:25.961 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:36:25.965 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T11:36:25.966 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:36:26.042 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.038+0000 7f876c251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.044 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.043+0000 7f876c251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.084 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.080+0000 7fe95a65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.084 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.080+0000 7fa08385a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.087 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.085+0000 7fa08385a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.087 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.085+0000 7fe95a65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.161 INFO:teuthology.orchestra.run.trial072.stdout:47244640265 2026-02-20T11:36:26.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640265 got 47244640265 for osd.2 2026-02-20T11:36:26.172 DEBUG:teuthology.parallel:result is None 2026-02-20T11:36:26.205 INFO:teuthology.orchestra.run.trial072.stdout:47244640265 2026-02-20T11:36:26.205 INFO:teuthology.orchestra.run.trial072.stdout:38654705675 2026-02-20T11:36:26.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640265 got 47244640265 for osd.1 2026-02-20T11:36:26.216 DEBUG:teuthology.parallel:result is None 2026-02-20T11:36:26.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705675 got 38654705675 for osd.0 2026-02-20T11:36:26.218 DEBUG:teuthology.parallel:result is None 2026-02-20T11:36:26.218 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T11:36:26.218 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:36:26.336 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.333+0000 7fb4df48b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.338 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.337+0000 7fb4df48b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.448 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:36:26.448 INFO:teuthology.orchestra.run.trial072.stderr:dumped all 2026-02-20T11:36:26.457 INFO:teuthology.orchestra.run.trial072.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-20T11:36:24.769681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2884,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497084,"statfs":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.076972"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.713533+0000","last_change":"2026-02-20T11:36:23.716350+0000","last_active":"2026-02-20T11:36:23.713533+0000","last_peered":"2026-02-20T11:36:23.713533+0000","last_clean":"2026-02-20T11:36:23.713533+0000","last_became_active":"2026-02-20T11:36:21.739004+0000","last_became_peered":"2026-02-20T11:36:21.739004+0000","last_unstale":"2026-02-20T11:36:23.713533+0000","last_undegraded":"2026-02-20T11:36:23.713533+0000","last_fullsized":"2026-02-20T11:36:23.713533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.715287+0000","last_change":"2026-02-20T11:36:23.717757+0000","last_active":"2026-02-20T11:36:23.715287+0000","last_peered":"2026-02-20T11:36:23.715287+0000","last_clean":"2026-02-20T11:36:23.715287+0000","last_became_active":"2026-02-20T11:36:21.740713+0000","last_became_peered":"2026-02-20T11:36:21.740713+0000","last_unstale":"2026-02-20T11:36:23.715287+0000","last_undegraded":"2026-02-20T11:36:23.715287+0000","last_fullsized":"2026-02-20T11:36:23.715287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.715512+0000","last_change":"2026-02-20T11:36:23.719961+0000","last_active":"2026-02-20T11:36:23.715512+0000","last_peered":"2026-02-20T11:36:23.715512+0000","last_clean":"2026-02-20T11:36:23.715512+0000","last_became_active":"2026-02-20T11:36:21.748377+0000","last_became_peered":"2026-02-20T11:36:21.748377+0000","last_unstale":"2026-02-20T11:36:23.715512+0000","last_undegraded":"2026-02-20T11:36:23.715512+0000","last_fullsized":"2026-02-20T11:36:23.715512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.714457+0000","last_change":"2026-02-20T11:36:23.717230+0000","last_active":"2026-02-20T11:36:23.714457+0000","last_peered":"2026-02-20T11:36:23.714457+0000","last_clean":"2026-02-20T11:36:23.714457+0000","last_became_active":"2026-02-20T11:36:21.745389+0000","last_became_peered":"2026-02-20T11:36:21.745389+0000","last_unstale":"2026-02-20T11:36:23.714457+0000","last_undegraded":"2026-02-20T11:36:23.714457+0000","last_fullsized":"2026-02-20T11:36:23.714457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.716373+0000","last_change":"2026-02-20T11:36:23.716373+0000","last_active":"2026-02-20T11:36:23.716373+0000","last_peered":"2026-02-20T11:36:23.716373+0000","last_clean":"2026-02-20T11:36:23.716373+0000","last_became_active":"2026-02-20T11:36:21.759920+0000","last_became_peered":"2026-02-20T11:36:21.759920+0000","last_unstale":"2026-02-20T11:36:23.716373+0000","last_undegraded":"2026-02-20T11:36:23.716373+0000","last_fullsized":"2026-02-20T11:36:23.716373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.713697+0000","last_change":"2026-02-20T11:36:23.716540+0000","last_active":"2026-02-20T11:36:23.713697+0000","last_peered":"2026-02-20T11:36:23.713697+0000","last_clean":"2026-02-20T11:36:23.713697+0000","last_became_active":"2026-02-20T11:36:21.752055+0000","last_became_peered":"2026-02-20T11:36:21.752055+0000","last_unstale":"2026-02-20T11:36:23.713697+0000","last_undegraded":"2026-02-20T11:36:23.713697+0000","last_fullsized":"2026-02-20T11:36:23.713697+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.711580+0000","last_change":"2026-02-20T11:36:23.714664+0000","last_active":"2026-02-20T11:36:23.711580+0000","last_peered":"2026-02-20T11:36:23.711580+0000","last_clean":"2026-02-20T11:36:23.711580+0000","last_became_active":"2026-02-20T11:36:21.760419+0000","last_became_peered":"2026-02-20T11:36:21.760419+0000","last_unstale":"2026-02-20T11:36:23.711580+0000","last_undegraded":"2026-02-20T11:36:23.711580+0000","last_fullsized":"2026-02-20T11:36:23.711580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.715008+0000","last_change":"2026-02-20T11:36:23.719908+0000","last_active":"2026-02-20T11:36:23.715008+0000","last_peered":"2026-02-20T11:36:23.715008+0000","last_clean":"2026-02-20T11:36:23.715008+0000","last_became_active":"2026-02-20T11:36:21.739009+0000","last_became_peered":"2026-02-20T11:36:21.739009+0000","last_unstale":"2026-02-20T11:36:23.715008+0000","last_undegraded":"2026-02-20T11:36:23.715008+0000","last_fullsized":"2026-02-20T11:36:23.715008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":90,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.712907+0000","last_change":"2026-02-20T11:36:18.711532+0000","last_active":"2026-02-20T11:36:23.712907+0000","last_peered":"2026-02-20T11:36:23.712907+0000","last_clean":"2026-02-20T11:36:23.712907+0000","last_became_active":"2026-02-20T11:36:18.702317+0000","last_became_peered":"2026-02-20T11:36:18.702317+0000","last_unstale":"2026-02-20T11:36:23.712907+0000","last_undegraded":"2026-02-20T11:36:23.712907+0000","last_fullsized":"2026-02-20T11:36:23.712907+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:17.668818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:17.668818+0000","last_clean_scrub_stamp":"2026-02-20T11:36:17.668818+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":11,"seq":47244640265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":372,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166284,"statfs":{"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1316,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165340,"statfs":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705675,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1196,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165460,"statfs":{"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:36:26.458 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:36:26.534 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.530+0000 7f1af2ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.536 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.534+0000 7f1af2ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.647 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:36:26.647 INFO:teuthology.orchestra.run.trial072.stderr:dumped all 2026-02-20T11:36:26.656 INFO:teuthology.orchestra.run.trial072.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-20T11:36:24.769681+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2884,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497084,"statfs":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.076972"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.713533+0000","last_change":"2026-02-20T11:36:23.716350+0000","last_active":"2026-02-20T11:36:23.713533+0000","last_peered":"2026-02-20T11:36:23.713533+0000","last_clean":"2026-02-20T11:36:23.713533+0000","last_became_active":"2026-02-20T11:36:21.739004+0000","last_became_peered":"2026-02-20T11:36:21.739004+0000","last_unstale":"2026-02-20T11:36:23.713533+0000","last_undegraded":"2026-02-20T11:36:23.713533+0000","last_fullsized":"2026-02-20T11:36:23.713533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.715287+0000","last_change":"2026-02-20T11:36:23.717757+0000","last_active":"2026-02-20T11:36:23.715287+0000","last_peered":"2026-02-20T11:36:23.715287+0000","last_clean":"2026-02-20T11:36:23.715287+0000","last_became_active":"2026-02-20T11:36:21.740713+0000","last_became_peered":"2026-02-20T11:36:21.740713+0000","last_unstale":"2026-02-20T11:36:23.715287+0000","last_undegraded":"2026-02-20T11:36:23.715287+0000","last_fullsized":"2026-02-20T11:36:23.715287+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.715512+0000","last_change":"2026-02-20T11:36:23.719961+0000","last_active":"2026-02-20T11:36:23.715512+0000","last_peered":"2026-02-20T11:36:23.715512+0000","last_clean":"2026-02-20T11:36:23.715512+0000","last_became_active":"2026-02-20T11:36:21.748377+0000","last_became_peered":"2026-02-20T11:36:21.748377+0000","last_unstale":"2026-02-20T11:36:23.715512+0000","last_undegraded":"2026-02-20T11:36:23.715512+0000","last_fullsized":"2026-02-20T11:36:23.715512+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.714457+0000","last_change":"2026-02-20T11:36:23.717230+0000","last_active":"2026-02-20T11:36:23.714457+0000","last_peered":"2026-02-20T11:36:23.714457+0000","last_clean":"2026-02-20T11:36:23.714457+0000","last_became_active":"2026-02-20T11:36:21.745389+0000","last_became_peered":"2026-02-20T11:36:21.745389+0000","last_unstale":"2026-02-20T11:36:23.714457+0000","last_undegraded":"2026-02-20T11:36:23.714457+0000","last_fullsized":"2026-02-20T11:36:23.714457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.716373+0000","last_change":"2026-02-20T11:36:23.716373+0000","last_active":"2026-02-20T11:36:23.716373+0000","last_peered":"2026-02-20T11:36:23.716373+0000","last_clean":"2026-02-20T11:36:23.716373+0000","last_became_active":"2026-02-20T11:36:21.759920+0000","last_became_peered":"2026-02-20T11:36:21.759920+0000","last_unstale":"2026-02-20T11:36:23.716373+0000","last_undegraded":"2026-02-20T11:36:23.716373+0000","last_fullsized":"2026-02-20T11:36:23.716373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.713697+0000","last_change":"2026-02-20T11:36:23.716540+0000","last_active":"2026-02-20T11:36:23.713697+0000","last_peered":"2026-02-20T11:36:23.713697+0000","last_clean":"2026-02-20T11:36:23.713697+0000","last_became_active":"2026-02-20T11:36:21.752055+0000","last_became_peered":"2026-02-20T11:36:21.752055+0000","last_unstale":"2026-02-20T11:36:23.713697+0000","last_undegraded":"2026-02-20T11:36:23.713697+0000","last_fullsized":"2026-02-20T11:36:23.713697+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.711580+0000","last_change":"2026-02-20T11:36:23.714664+0000","last_active":"2026-02-20T11:36:23.711580+0000","last_peered":"2026-02-20T11:36:23.711580+0000","last_clean":"2026-02-20T11:36:23.711580+0000","last_became_active":"2026-02-20T11:36:21.760419+0000","last_became_peered":"2026-02-20T11:36:21.760419+0000","last_unstale":"2026-02-20T11:36:23.711580+0000","last_undegraded":"2026-02-20T11:36:23.711580+0000","last_fullsized":"2026-02-20T11:36:23.711580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.715008+0000","last_change":"2026-02-20T11:36:23.719908+0000","last_active":"2026-02-20T11:36:23.715008+0000","last_peered":"2026-02-20T11:36:23.715008+0000","last_clean":"2026-02-20T11:36:23.715008+0000","last_became_active":"2026-02-20T11:36:21.739009+0000","last_became_peered":"2026-02-20T11:36:21.739009+0000","last_unstale":"2026-02-20T11:36:23.715008+0000","last_undegraded":"2026-02-20T11:36:23.715008+0000","last_fullsized":"2026-02-20T11:36:23.715008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:20.679046+0000","last_clean_scrub_stamp":"2026-02-20T11:36:20.679046+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":90,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:36:23.712907+0000","last_change":"2026-02-20T11:36:18.711532+0000","last_active":"2026-02-20T11:36:23.712907+0000","last_peered":"2026-02-20T11:36:23.712907+0000","last_clean":"2026-02-20T11:36:23.712907+0000","last_became_active":"2026-02-20T11:36:18.702317+0000","last_became_peered":"2026-02-20T11:36:18.702317+0000","last_unstale":"2026-02-20T11:36:23.712907+0000","last_undegraded":"2026-02-20T11:36:23.712907+0000","last_fullsized":"2026-02-20T11:36:23.712907+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:36:17.668818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:36:17.668818+0000","last_clean_scrub_stamp":"2026-02-20T11:36:17.668818+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":11,"seq":47244640265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":372,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166284,"statfs":{"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1316,"kb_used_data":1316,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165340,"statfs":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705675,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1196,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165460,"statfs":{"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:36:26.657 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-20T11:36:26.657 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-20T11:36:26.657 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-20T11:36:26.658 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-20T11:36:26.736 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.732+0000 7f455bc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.738 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T11:36:26.736+0000 7f455bc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:36:26.877 INFO:teuthology.orchestra.run.trial072.stdout: 2026-02-20T11:36:26.877 INFO:teuthology.orchestra.run.trial072.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-20T11:36:26.887 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-20T11:36:26.887 INFO:teuthology.run_tasks:Running task workunit... 2026-02-20T11:36:26.914 INFO:tasks.workunit:Pulling workunits from ref 445d5c9ba18a4040e70e25303230c36c77773acb 2026-02-20T11:36:26.914 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-20T11:36:26.914 DEBUG:teuthology.orchestra.run.trial072:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-20T11:36:26.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:36:26.931 INFO:teuthology.orchestra.run.trial072.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-20T11:36:26.932 DEBUG:teuthology.orchestra.run.trial072:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-20T11:36:26.988 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-20T11:36:26.988 DEBUG:teuthology.orchestra.run.trial072:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-20T11:36:27.045 INFO:tasks.workunit:timeout=3h 2026-02-20T11:36:27.045 INFO:tasks.workunit:cleanup=True 2026-02-20T11:36:27.045 DEBUG:teuthology.orchestra.run.trial072:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 445d5c9ba18a4040e70e25303230c36c77773acb 2026-02-20T11:36:27.108 INFO:tasks.workunit.client.0.trial072.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-20T11:36:48.559 INFO:tasks.workunit.client.0.trial072.stderr:Note: switching to '445d5c9ba18a4040e70e25303230c36c77773acb'. 2026-02-20T11:36:48.559 INFO:tasks.workunit.client.0.trial072.stderr: 2026-02-20T11:36:48.559 INFO:tasks.workunit.client.0.trial072.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr:state without impacting any branches by switching back to a branch. 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr: 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr: 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr: git switch -c 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr: 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr:Or undo this operation with: 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr: 2026-02-20T11:36:48.560 INFO:tasks.workunit.client.0.trial072.stderr: git switch - 2026-02-20T11:36:48.561 INFO:tasks.workunit.client.0.trial072.stderr: 2026-02-20T11:36:48.561 INFO:tasks.workunit.client.0.trial072.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-20T11:36:48.561 INFO:tasks.workunit.client.0.trial072.stderr: 2026-02-20T11:36:48.561 INFO:tasks.workunit.client.0.trial072.stderr:HEAD is now at 445d5c9ba18 Merge PR #66907 into main 2026-02-20T11:36:48.562 DEBUG:teuthology.orchestra.run.trial072:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-20T11:36:48.619 INFO:tasks.workunit.client.0.trial072.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-20T11:36:48.621 INFO:tasks.workunit.client.0.trial072.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-20T11:36:48.622 INFO:tasks.workunit.client.0.trial072.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-20T11:36:48.710 INFO:tasks.workunit.client.0.trial072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-20T11:36:48.743 INFO:tasks.workunit.client.0.trial072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-20T11:36:48.771 INFO:tasks.workunit.client.0.trial072.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-20T11:36:48.773 INFO:tasks.workunit.client.0.trial072.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-20T11:36:48.773 INFO:tasks.workunit.client.0.trial072.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-20T11:36:48.798 INFO:tasks.workunit.client.0.trial072.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-20T11:36:48.802 DEBUG:teuthology.orchestra.run.trial072:> set -ex 2026-02-20T11:36:48.802 DEBUG:teuthology.orchestra.run.trial072:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-20T11:36:48.860 INFO:tasks.workunit:Running workunits matching rbd/test_librbd.sh on client.0... 2026-02-20T11:36:48.863 INFO:tasks.workunit:Running workunit rbd/test_librbd.sh... 2026-02-20T11:36:48.863 DEBUG:teuthology.orchestra.run.trial072:workunit test rbd/test_librbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=445d5c9ba18a4040e70e25303230c36c77773acb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh 2026-02-20T11:36:48.954 INFO:tasks.workunit.client.0.trial072.stdout:seed 41912 2026-02-20T11:36:48.954 INFO:tasks.workunit.client.0.trial072.stdout:[==========] Running 500 tests from 24 test suites. 2026-02-20T11:36:48.954 INFO:tasks.workunit.client.0.trial072.stdout:[----------] Global test environment set-up. 2026-02-20T11:36:48.955 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 1 test from TestJournalStress 2026-02-20T11:36:50.837 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalStress.DiscardWithPruneWriteOverlap 2026-02-20T11:36:52.925 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Stress.cc:37: Skipped 2026-02-20T11:36:52.926 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:52.926 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:52.926 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalStress.DiscardWithPruneWriteOverlap (2087 ms) 2026-02-20T11:36:53.844 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 1 test from TestJournalStress (2087 ms total) 2026-02-20T11:36:53.845 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:53.845 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 16 tests from TestJournalReplay 2026-02-20T11:36:55.849 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.AioDiscardEvent 2026-02-20T11:36:57.935 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:111: Skipped 2026-02-20T11:36:57.935 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:57.935 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:57.935 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.AioDiscardEvent (2086 ms) 2026-02-20T11:36:57.935 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.AioWriteEvent 2026-02-20T11:36:57.993 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:202: Skipped 2026-02-20T11:36:57.993 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:57.994 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:57.994 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.AioWriteEvent (58 ms) 2026-02-20T11:36:57.994 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.AioFlushEvent 2026-02-20T11:36:58.043 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:263: Skipped 2026-02-20T11:36:58.043 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.043 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.043 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.AioFlushEvent (49 ms) 2026-02-20T11:36:58.043 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.SnapCreate 2026-02-20T11:36:58.094 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:309: Skipped 2026-02-20T11:36:58.094 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.094 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.094 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.SnapCreate (49 ms) 2026-02-20T11:36:58.094 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.SnapProtect 2026-02-20T11:36:58.144 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:350: Skipped 2026-02-20T11:36:58.144 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.144 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.144 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.SnapProtect (50 ms) 2026-02-20T11:36:58.144 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.SnapUnprotect 2026-02-20T11:36:58.195 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:396: Skipped 2026-02-20T11:36:58.195 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.195 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.195 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.SnapUnprotect (50 ms) 2026-02-20T11:36:58.196 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.SnapRename 2026-02-20T11:36:58.245 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:452: Skipped 2026-02-20T11:36:58.246 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.246 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.246 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.SnapRename (50 ms) 2026-02-20T11:36:58.246 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.SnapRollback 2026-02-20T11:36:58.295 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:502: Skipped 2026-02-20T11:36:58.295 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.295 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.295 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.SnapRollback (49 ms) 2026-02-20T11:36:58.295 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.SnapRemove 2026-02-20T11:36:58.357 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:543: Skipped 2026-02-20T11:36:58.357 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.357 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.358 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.SnapRemove (62 ms) 2026-02-20T11:36:58.358 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.Rename 2026-02-20T11:36:58.408 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:592: Skipped 2026-02-20T11:36:58.408 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.408 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.408 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.Rename (51 ms) 2026-02-20T11:36:58.409 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.Resize 2026-02-20T11:36:58.461 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:626: Skipped 2026-02-20T11:36:58.461 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.461 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.461 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.Resize (52 ms) 2026-02-20T11:36:58.462 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.Flatten 2026-02-20T11:36:58.511 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:659: Skipped 2026-02-20T11:36:58.511 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.511 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.511 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.Flatten (49 ms) 2026-02-20T11:36:58.512 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.UpdateFeatures 2026-02-20T11:36:58.563 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:706: Skipped 2026-02-20T11:36:58.563 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.563 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.563 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.UpdateFeatures (51 ms) 2026-02-20T11:36:58.563 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.MetadataSet 2026-02-20T11:36:58.612 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:743: Skipped 2026-02-20T11:36:58.612 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.613 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.613 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.MetadataSet (49 ms) 2026-02-20T11:36:58.613 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.MetadataRemove 2026-02-20T11:36:58.662 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:783: Skipped 2026-02-20T11:36:58.662 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.662 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.662 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.MetadataRemove (49 ms) 2026-02-20T11:36:58.662 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalReplay.ObjectPosition 2026-02-20T11:36:58.715 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Replay.cc:826: Skipped 2026-02-20T11:36:58.715 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:36:58.715 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.715 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalReplay.ObjectPosition (52 ms) 2026-02-20T11:36:58.868 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 16 tests from TestJournalReplay (2865 ms total) 2026-02-20T11:36:58.868 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:36:58.868 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 4 tests from TestJournalEntries 2026-02-20T11:37:00.873 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalEntries.AioWrite 2026-02-20T11:37:02.959 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Entries.cc:117: Skipped 2026-02-20T11:37:02.959 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:02.960 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:02.960 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalEntries.AioWrite (2086 ms) 2026-02-20T11:37:02.960 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalEntries.AioDiscard 2026-02-20T11:37:03.014 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Entries.cc:162: Skipped 2026-02-20T11:37:03.015 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:03.015 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:03.015 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalEntries.AioDiscard (55 ms) 2026-02-20T11:37:03.015 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalEntries.AioDiscardWithPrune 2026-02-20T11:37:03.065 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Entries.cc:199: Skipped 2026-02-20T11:37:03.065 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:03.065 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:03.065 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalEntries.AioDiscardWithPrune (50 ms) 2026-02-20T11:37:03.065 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestJournalEntries.AioFlush 2026-02-20T11:37:03.118 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/journal/test_Entries.cc:262: Skipped 2026-02-20T11:37:03.118 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:03.119 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:03.119 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestJournalEntries.AioFlush (53 ms) 2026-02-20T11:37:03.882 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 4 tests from TestJournalEntries (2245 ms total) 2026-02-20T11:37:03.883 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:03.883 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 2 tests from TestTrash 2026-02-20T11:37:05.890 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestTrash.UserRemovingSource 2026-02-20T11:37:08.462 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestTrash.UserRemovingSource (2572 ms) 2026-02-20T11:37:08.462 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestTrash.RestoreMirroringSource 2026-02-20T11:37:08.787 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestTrash.RestoreMirroringSource (324 ms) 2026-02-20T11:37:08.903 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 2 tests from TestTrash (2896 ms total) 2026-02-20T11:37:08.904 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:08.904 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 1 test from TestOperations 2026-02-20T11:37:10.914 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestOperations.DisableJournalingCorrupt 2026-02-20T11:37:12.999 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/test_Operations.cc:18: Skipped 2026-02-20T11:37:13.000 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:13.000 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:13.000 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestOperations.DisableJournalingCorrupt (2085 ms) 2026-02-20T11:37:13.924 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 1 test from TestOperations (2085 ms total) 2026-02-20T11:37:13.924 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:13.924 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 4 tests from TestObjectMap 2026-02-20T11:37:15.934 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestObjectMap.RefreshInvalidatesWhenCorrupt 2026-02-20T11:37:18.163 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestObjectMap.RefreshInvalidatesWhenCorrupt (2229 ms) 2026-02-20T11:37:18.163 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestObjectMap.RefreshInvalidatesWhenTooSmall 2026-02-20T11:37:18.363 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestObjectMap.RefreshInvalidatesWhenTooSmall (199 ms) 2026-02-20T11:37:18.363 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestObjectMap.InvalidateFlagOnDisk 2026-02-20T11:37:18.615 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestObjectMap.InvalidateFlagOnDisk (252 ms) 2026-02-20T11:37:18.615 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestObjectMap.AcquireLockInvalidatesWhenTooSmall 2026-02-20T11:37:18.830 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestObjectMap.AcquireLockInvalidatesWhenTooSmall (214 ms) 2026-02-20T11:37:18.830 INFO:tasks.workunit.client.0.trial072.stdout:[ DISABLED ] TestObjectMap.DISABLED_StressTest 2026-02-20T11:37:18.951 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 4 tests from TestObjectMap (2895 ms total) 2026-02-20T11:37:18.951 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:18.951 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 2 tests from TestMirroringWatcher 2026-02-20T11:37:20.954 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMirroringWatcher.ModeUpdated 2026-02-20T11:37:23.129 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMirroringWatcher.ModeUpdated (2175 ms) 2026-02-20T11:37:23.129 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMirroringWatcher.ImageStatusUpdated 2026-02-20T11:37:23.259 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMirroringWatcher.ImageStatusUpdated (130 ms) 2026-02-20T11:37:23.965 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 2 tests from TestMirroringWatcher (2305 ms total) 2026-02-20T11:37:23.965 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:23.965 INFO:tasks.workunit.client.0.trial072.stdout:[----------] 58 tests from TestMigration 2026-02-20T11:37:26.976 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.Empty 2026-02-20T11:37:29.217 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:29.315 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:30.252 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:30.822 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:37:31.242 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.Empty (4266 ms) 2026-02-20T11:37:31.243 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.OtherName 2026-02-20T11:37:31.457 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:31.561 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:32.403 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:32.933 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:37:33.360 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.OtherName (2117 ms) 2026-02-20T11:37:33.360 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.OtherPool 2026-02-20T11:37:33.579 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:33.675 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:35.640 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:36.195 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:37:36.629 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.OtherPool (3269 ms) 2026-02-20T11:37:36.630 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.OtherNamespace 2026-02-20T11:37:36.848 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:36.975 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:37.840 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:38.364 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:37:38.816 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.OtherNamespace (2186 ms) 2026-02-20T11:37:38.816 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.DataPool 2026-02-20T11:37:39.034 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:39.133 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:40.105 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:40.710 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:37:41.169 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.DataPool (2352 ms) 2026-02-20T11:37:41.169 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.AbortAfterPrepare 2026-02-20T11:37:41.396 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:41.506 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:42.531 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:37:43.108 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.AbortAfterPrepare (1939 ms) 2026-02-20T11:37:43.108 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.AbortAfterFailedPrepare 2026-02-20T11:37:43.318 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:43.419 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:44.001 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.AbortAfterFailedPrepare (892 ms) 2026-02-20T11:37:44.001 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.AbortAfterExecute 2026-02-20T11:37:44.253 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:44.356 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:45.339 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:45.946 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:37:46.536 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.AbortAfterExecute (2535 ms) 2026-02-20T11:37:46.536 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.OtherPoolAbortAfterExecute 2026-02-20T11:37:46.749 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:46.849 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:47.730 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:48.277 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:37:48.876 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.OtherPoolAbortAfterExecute (2340 ms) 2026-02-20T11:37:48.876 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.OtherNamespaceAbortAfterExecute 2026-02-20T11:37:49.091 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:49.226 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:50.092 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:50.613 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:37:51.266 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.OtherNamespaceAbortAfterExecute (2389 ms) 2026-02-20T11:37:51.266 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.MirroringSamePool 2026-02-20T11:37:51.482 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:51.583 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/test_Migration.cc:648: Skipped 2026-02-20T11:37:51.584 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:51.584 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:51.649 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestMigration.MirroringSamePool (383 ms) 2026-02-20T11:37:51.649 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.MirroringAbort 2026-02-20T11:37:51.860 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:51.962 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/test_Migration.cc:666: Skipped 2026-02-20T11:37:51.962 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:51.962 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:52.022 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestMigration.MirroringAbort (372 ms) 2026-02-20T11:37:52.022 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.MirroringOtherPoolDisabled 2026-02-20T11:37:52.273 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:52.374 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/test_Migration.cc:689: Skipped 2026-02-20T11:37:52.374 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:52.374 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:52.433 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestMigration.MirroringOtherPoolDisabled (411 ms) 2026-02-20T11:37:52.433 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.MirroringOtherPoolEnabled 2026-02-20T11:37:52.656 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:52.769 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/test_Migration.cc:707: Skipped 2026-02-20T11:37:52.769 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:52.769 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:52.830 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestMigration.MirroringOtherPoolEnabled (396 ms) 2026-02-20T11:37:52.830 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.MirroringPool 2026-02-20T11:37:53.061 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:53.162 INFO:tasks.workunit.client.0.trial072.stdout:/ceph/rpmbuild/BUILD/ceph-20.3.0-5357-g0c3148ca/src/test/librbd/test_Migration.cc:727: Skipped 2026-02-20T11:37:53.163 INFO:tasks.workunit.client.0.trial072.stdout:Skipping due to unmet REQUIRE 2026-02-20T11:37:53.163 INFO:tasks.workunit.client.0.trial072.stdout: 2026-02-20T11:37:53.223 INFO:tasks.workunit.client.0.trial072.stdout:[ SKIPPED ] TestMigration.MirroringPool (392 ms) 2026-02-20T11:37:53.223 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.Group 2026-02-20T11:37:53.435 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:53.633 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:54.658 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:37:55.243 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:37:55.792 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.Group (2569 ms) 2026-02-20T11:37:55.792 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.GroupAbort 2026-02-20T11:37:56.019 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:56.213 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:57.293 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:37:58.039 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.GroupAbort (2246 ms) 2026-02-20T11:37:58.039 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.NoSnaps 2026-02-20T11:37:58.284 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:37:58.385 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:37:58.571 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:37:58.727 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:37:59.771 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:38:00.532 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:38:01.105 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.NoSnaps (3066 ms) 2026-02-20T11:38:01.105 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.NoSnapsOtherPool 2026-02-20T11:38:01.323 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:38:01.438 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:38:01.625 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:38:01.763 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:38:01.766 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:38:01.839 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:38:02.765 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:38:03.446 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:38:04.013 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.NoSnapsOtherPool (2908 ms) 2026-02-20T11:38:04.013 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.NoSnapsDataPool 2026-02-20T11:38:04.242 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:38:04.361 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:38:04.562 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:38:04.713 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:38:05.775 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:38:06.569 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:38:07.150 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.NoSnapsDataPool (3137 ms) 2026-02-20T11:38:07.151 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.NoSnapsShrinkAfterPrepare 2026-02-20T11:38:07.418 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:38:07.527 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:38:07.746 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:38:07.888 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:38:41.776 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-20T11:38:47.179 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2026-02-20T11:38:52.582 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~11s 2026-02-20T11:38:57.985 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~16s 2026-02-20T11:39:03.388 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~22s 2026-02-20T11:39:08.791 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~27s 2026-02-20T11:39:12.261 INFO:tasks.workunit.client.0.trial072.stdout:resize: 41943040 2026-02-20T11:39:12.347 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:39:13.133 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:39:13.724 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.NoSnapsShrinkAfterPrepare (66573 ms) 2026-02-20T11:39:13.724 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.NoSnapsShrinkToZeroBeforePrepare 2026-02-20T11:39:13.941 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:14.033 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:39:14.195 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~32s 2026-02-20T11:39:14.232 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:39:14.392 INFO:tasks.workunit.client.0.trial072.stdout:resize: 0 2026-02-20T11:39:14.554 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:39:15.436 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:39:16.000 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:39:16.390 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.NoSnapsShrinkToZeroBeforePrepare (2665 ms) 2026-02-20T11:39:16.390 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.NoSnapsShrinkToZeroAfterPrepare 2026-02-20T11:39:16.613 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:16.718 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:39:16.917 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:39:17.190 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:39:18.262 INFO:tasks.workunit.client.0.trial072.stdout:resize: 0 2026-02-20T11:39:18.406 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:39:18.988 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:39:19.497 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.NoSnapsShrinkToZeroAfterPrepare (3107 ms) 2026-02-20T11:39:19.497 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.NoSnapsExpandAfterPrepare 2026-02-20T11:39:19.597 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~38s 2026-02-20T11:39:19.714 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:19.821 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:39:20.023 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:39:20.197 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:39:21.251 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:39:21.324 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:39:22.099 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:39:22.708 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.NoSnapsExpandAfterPrepare (3211 ms) 2026-02-20T11:39:22.708 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.NoSnapsSnapAfterPrepare 2026-02-20T11:39:22.934 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:23.027 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:39:23.239 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:39:23.382 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:39:24.418 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: after_prepare_snap 2026-02-20T11:39:25.000 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~43s 2026-02-20T11:39:26.457 INFO:tasks.workunit.client.0.trial072.stdout:resize: 41943040 2026-02-20T11:39:26.657 INFO:tasks.workunit.client.0.trial072.stdout:write: * 0~1000 2026-02-20T11:39:26.746 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:39:27.554 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:39:28.226 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.NoSnapsSnapAfterPrepare (5518 ms) 2026-02-20T11:39:28.227 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.Snaps 2026-02-20T11:39:28.457 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:28.551 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:39:28.731 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:39:30.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~49s 2026-02-20T11:39:30.480 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:39:30.567 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:39:30.567 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:39:30.567 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:39:32.491 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:39:32.587 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:39:32.587 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:39:32.665 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:39:32.759 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:39:34.487 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:34.743 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:39:34.877 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:39:35.806 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~54s 2026-02-20T11:39:38.967 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:39:40.039 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:39:41.209 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~60s 2026-02-20T11:39:43.111 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.Snaps (14884 ms) 2026-02-20T11:39:43.111 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsOtherPool 2026-02-20T11:39:43.355 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:43.457 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:39:43.676 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:39:45.407 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:39:45.511 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:39:45.511 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:39:45.511 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:39:46.611 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~65s 2026-02-20T11:39:47.530 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:39:47.620 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:39:47.620 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:39:47.730 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:39:47.832 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:39:49.532 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:49.833 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:39:49.970 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:39:49.974 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:39:50.204 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:39:52.014 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~70s 2026-02-20T11:39:54.887 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:39:56.001 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:39:57.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~76s 2026-02-20T11:39:59.170 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsOtherPool (16058 ms) 2026-02-20T11:39:59.170 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsDataPool 2026-02-20T11:39:59.414 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:39:59.525 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:39:59.722 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:40:01.582 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:40:01.698 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:40:01.699 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:40:01.699 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:40:02.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~81s 2026-02-20T11:40:03.581 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:40:03.678 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:40:03.678 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:40:03.781 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:40:03.896 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:40:05.611 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:40:05.864 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:40:06.002 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:40:08.222 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~87s 2026-02-20T11:40:10.141 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:40:11.277 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:40:13.625 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~92s 2026-02-20T11:40:15.234 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsDataPool (16064 ms) 2026-02-20T11:40:15.235 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsShrinkAfterPrepare 2026-02-20T11:40:15.471 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:40:15.572 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:40:15.771 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:40:17.626 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:40:17.761 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:40:17.761 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:40:17.761 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:40:19.028 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~97s 2026-02-20T11:40:19.629 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:40:19.733 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:40:19.733 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:40:19.824 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:40:19.960 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:40:21.635 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:40:21.884 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:40:22.035 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:40:24.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~103s 2026-02-20T11:40:25.923 INFO:tasks.workunit.client.0.trial072.stdout:resize: 41943040 2026-02-20T11:40:26.148 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:40:27.248 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:40:29.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~108s 2026-02-20T11:40:30.622 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsShrinkAfterPrepare (15388 ms) 2026-02-20T11:40:30.623 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsShrinkToZeroBeforePrepare 2026-02-20T11:40:30.860 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:40:30.973 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:40:31.164 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:40:32.968 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:40:33.046 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:40:33.047 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:40:33.047 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:40:34.978 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:40:35.074 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:40:35.075 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:40:35.178 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:40:35.237 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~114s 2026-02-20T11:40:35.285 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:40:37.423 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:40:37.691 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:40:37.830 INFO:tasks.workunit.client.0.trial072.stdout:resize: 0 2026-02-20T11:40:38.054 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:40:40.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~119s 2026-02-20T11:40:42.781 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:40:44.011 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:40:46.043 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~124s 2026-02-20T11:40:47.873 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsShrinkToZeroBeforePrepare (17250 ms) 2026-02-20T11:40:47.873 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsShrinkToZeroAfterPrepare 2026-02-20T11:40:48.103 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:40:48.207 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:40:48.405 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:40:50.463 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:40:50.546 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:40:50.546 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:40:50.546 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:40:51.446 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~130s 2026-02-20T11:40:52.467 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:40:52.555 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:40:52.556 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:40:52.654 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:40:52.755 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:40:54.472 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:40:54.739 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:40:54.867 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:40:56.848 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~135s 2026-02-20T11:40:58.917 INFO:tasks.workunit.client.0.trial072.stdout:resize: 0 2026-02-20T11:40:59.466 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:41:00.521 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:41:02.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~141s 2026-02-20T11:41:04.048 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsShrinkToZeroAfterPrepare (16175 ms) 2026-02-20T11:41:04.048 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsExpandAfterPrepare 2026-02-20T11:41:04.283 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:41:04.408 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:41:04.621 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:41:06.495 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:41:06.592 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:41:06.592 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:41:06.592 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:41:07.654 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~146s 2026-02-20T11:41:08.513 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:41:08.605 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:41:08.605 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:41:08.702 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:41:08.806 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:41:10.512 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:41:10.768 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:41:10.914 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:41:13.056 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~151s 2026-02-20T11:41:15.070 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:41:15.175 INFO:tasks.workunit.client.0.trial072.stdout:write: * 83886080~1000 2026-02-20T11:41:15.375 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:41:16.626 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:41:18.459 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~157s 2026-02-20T11:41:20.349 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsExpandAfterPrepare (16301 ms) 2026-02-20T11:41:20.350 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsExpandAfterPrepare2 2026-02-20T11:41:20.571 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:41:20.665 INFO:tasks.workunit.client.0.trial072.stdout:write: X 41943040~10 2026-02-20T11:41:20.771 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:41:22.543 INFO:tasks.workunit.client.0.trial072.stdout:resize: 41943040 2026-02-20T11:41:22.762 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:41:23.861 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~162s 2026-02-20T11:41:24.403 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:41:24.492 INFO:tasks.workunit.client.0.trial072.stdout:write: Y 41943040~5 2026-02-20T11:41:24.942 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:41:25.625 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:41:26.776 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsExpandAfterPrepare2 (6426 ms) 2026-02-20T11:41:26.777 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsSnapAfterPrepare 2026-02-20T11:41:27.010 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:41:27.104 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:41:27.274 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:41:28.555 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:41:28.628 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:41:28.629 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:41:28.629 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:41:29.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~168s 2026-02-20T11:41:30.426 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:41:30.511 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:41:30.511 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:41:30.595 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:41:30.680 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:41:32.567 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:41:32.810 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:41:32.940 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:41:34.667 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~173s 2026-02-20T11:41:37.443 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: after_prepare_snap 2026-02-20T11:41:38.591 INFO:tasks.workunit.client.0.trial072.stdout:resize: 41943040 2026-02-20T11:41:38.958 INFO:tasks.workunit.client.0.trial072.stdout:write: * 0~1000 2026-02-20T11:41:39.075 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:41:40.070 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~178s 2026-02-20T11:41:40.352 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:41:44.233 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsSnapAfterPrepare (17456 ms) 2026-02-20T11:41:44.233 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapsSnapExpandAfterPrepare 2026-02-20T11:41:44.490 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:41:44.601 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:41:44.833 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:41:45.473 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~184s 2026-02-20T11:41:46.611 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:41:46.705 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:41:46.705 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:41:46.705 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:41:48.617 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:41:48.711 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:41:48.711 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:41:48.812 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:41:48.915 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:41:50.633 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:41:50.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~189s 2026-02-20T11:41:50.913 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:41:51.056 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:41:55.152 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: after_prepare_snap 2026-02-20T11:41:56.280 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~195s 2026-02-20T11:41:56.637 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:41:56.798 INFO:tasks.workunit.client.0.trial072.stdout:write: * 83886080~1000 2026-02-20T11:41:56.925 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:41:58.137 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:42:01.235 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapsSnapExpandAfterPrepare (17001 ms) 2026-02-20T11:42:01.235 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.Clone 2026-02-20T11:42:01.469 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:01.574 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~8388609 2026-02-20T11:42:01.683 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~200s 2026-02-20T11:42:01.759 INFO:tasks.workunit.client.0.trial072.stdout:write: X 16777218~8388609 2026-02-20T11:42:01.896 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap 2026-02-20T11:42:03.446 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap 2026-02-20T11:42:03.583 INFO:tasks.workunit.client.0.trial072.stdout:clone image105 -> image108 2026-02-20T11:42:04.393 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:42:04.525 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:42:04.546 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap 2026-02-20T11:42:06.453 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap 2026-02-20T11:42:06.583 INFO:tasks.workunit.client.0.trial072.stdout:clone image108 -> image110 2026-02-20T11:42:07.086 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~205s 2026-02-20T11:42:07.580 INFO:tasks.workunit.client.0.trial072.stdout:write: X 1000~1000 2026-02-20T11:42:07.718 INFO:tasks.workunit.client.0.trial072.stdout:discard: 2010~1000 2026-02-20T11:42:07.735 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:42:09.793 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:42:11.182 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:42:12.246 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.Clone (11011 ms) 2026-02-20T11:42:12.247 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneParent 2026-02-20T11:42:12.485 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:12.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~211s 2026-02-20T11:42:12.596 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap 2026-02-20T11:42:14.435 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap 2026-02-20T11:42:14.574 INFO:tasks.workunit.client.0.trial072.stdout:clone image111 -> image114 2026-02-20T11:42:15.375 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:42:16.532 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:42:17.385 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.CloneParent (5138 ms) 2026-02-20T11:42:17.385 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneUpdateAfterPrepare 2026-02-20T11:42:17.629 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:17.732 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~10 2026-02-20T11:42:17.835 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap 2026-02-20T11:42:17.891 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~216s 2026-02-20T11:42:19.432 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap 2026-02-20T11:42:19.571 INFO:tasks.workunit.client.0.trial072.stdout:clone image115 -> image118 2026-02-20T11:42:20.397 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:42:21.585 INFO:tasks.workunit.client.0.trial072.stdout:write: Y 0~1 2026-02-20T11:42:21.941 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:42:22.634 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:42:23.294 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~222s 2026-02-20T11:42:23.544 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.CloneUpdateAfterPrepare (6160 ms) 2026-02-20T11:42:23.545 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneFlatten 2026-02-20T11:42:23.771 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:23.879 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~10 2026-02-20T11:42:23.985 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap 2026-02-20T11:42:25.487 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap 2026-02-20T11:42:25.619 INFO:tasks.workunit.client.0.trial072.stdout:clone image119 -> image122 2026-02-20T11:42:26.407 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:42:27.298 INFO:tasks.workunit.client.0.trial072.stdout:write: Y 0~1 2026-02-20T11:42:27.598 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:42:28.526 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:42:28.696 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~227s 2026-02-20T11:42:29.292 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.CloneFlatten (5747 ms) 2026-02-20T11:42:29.292 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.TriggerAssertSnapcSeq 2026-02-20T11:42:29.526 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:29.633 INFO:tasks.workunit.client.0.trial072.stdout:write: A 41943040~10 2026-02-20T11:42:29.737 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:42:31.546 INFO:tasks.workunit.client.0.trial072.stdout:write: B 41943041~10 2026-02-20T11:42:31.617 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:42:33.165 INFO:tasks.workunit.client.0.trial072.stdout:write: C 41943042~10 2026-02-20T11:42:33.311 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:42:34.099 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~232s 2026-02-20T11:42:34.553 INFO:tasks.workunit.client.0.trial072.stdout:resize: 41943040 2026-02-20T11:42:34.872 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:42:35.518 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:42:36.910 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.TriggerAssertSnapcSeq (7617 ms) 2026-02-20T11:42:36.910 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.SnapTrimBeforePrepare 2026-02-20T11:42:37.152 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:37.264 INFO:tasks.workunit.client.0.trial072.stdout:write: A 41943040~10 2026-02-20T11:42:37.376 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:42:38.568 INFO:tasks.workunit.client.0.trial072.stdout:resize: 41943040 2026-02-20T11:42:38.811 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:42:39.503 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~238s 2026-02-20T11:42:40.182 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:40.277 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:42:41.573 INFO:tasks.workunit.client.0.trial072.stdout:write: B 41943040~10 2026-02-20T11:42:41.679 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap4 2026-02-20T11:42:43.585 INFO:tasks.workunit.client.0.trial072.stdout:resize: 41943040 2026-02-20T11:42:43.845 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:42:44.479 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:42:44.906 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~243s 2026-02-20T11:42:45.952 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.SnapTrimBeforePrepare (9042 ms) 2026-02-20T11:42:45.952 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.AbortInUseImage 2026-02-20T11:42:46.213 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:46.321 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:42:47.749 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.AbortInUseImage (1796 ms) 2026-02-20T11:42:47.749 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.AbortWithoutSnapshots 2026-02-20T11:42:47.996 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:48.103 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:42:48.306 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:42:48.436 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:42:49.527 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:42:49.648 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:42:49.755 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:42:50.309 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~249s 2026-02-20T11:42:50.535 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.AbortWithoutSnapshots (2786 ms) 2026-02-20T11:42:50.536 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.AbortWithSnapshots 2026-02-20T11:42:50.772 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:50.876 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:42:51.066 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:42:52.598 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 8388609~8388609 2026-02-20T11:42:52.683 INFO:tasks.workunit.client.0.trial072.stdout:write: X 0~1000 2026-02-20T11:42:52.683 INFO:tasks.workunit.client.0.trial072.stdout:discard: 1010~1000 2026-02-20T11:42:52.683 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap2 2026-02-20T11:42:54.448 INFO:tasks.workunit.client.0.trial072.stdout:write: X 8388609~1000 2026-02-20T11:42:54.546 INFO:tasks.workunit.client.0.trial072.stdout:discard: 16777228~1000 2026-02-20T11:42:54.546 INFO:tasks.workunit.client.0.trial072.stdout:resize: 167772160 2026-02-20T11:42:54.640 INFO:tasks.workunit.client.0.trial072.stdout:write: 2 83886079~8388609 2026-02-20T11:42:54.748 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap3 2026-02-20T11:42:55.712 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~254s 2026-02-20T11:42:56.611 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:42:56.874 INFO:tasks.workunit.client.0.trial072.stdout:discard: 83886079~1 2026-02-20T11:42:57.018 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:43:01.115 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~259s 2026-02-20T11:43:01.126 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:43:01.442 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:43:02.628 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 0~8388609 2026-02-20T11:43:02.708 INFO:tasks.workunit.client.0.trial072.stdout:write: 1 16777218~8388609 2026-02-20T11:43:02.964 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:43:06.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~265s 2026-02-20T11:43:08.369 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.AbortWithSnapshots (17833 ms) 2026-02-20T11:43:08.369 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneV1Parent 2026-02-20T11:43:08.606 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:43:08.712 INFO:tasks.workunit.client.0.trial072.stdout:write: A 0~10 2026-02-20T11:43:08.816 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:43:10.663 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap1 2026-02-20T11:43:11.174 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:43:11.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~270s 2026-02-20T11:43:13.843 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:43:14.638 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:43:16.050 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.CloneV1Parent (7681 ms) 2026-02-20T11:43:16.051 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneV2Parent 2026-02-20T11:43:16.286 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:43:16.389 INFO:tasks.workunit.client.0.trial072.stdout:write: A 0~10 2026-02-20T11:43:16.498 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:43:17.325 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~276s 2026-02-20T11:43:17.679 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap1 2026-02-20T11:43:18.192 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:43:20.515 INFO:tasks.workunit.client.0.trial072.stdout:migration_execute 2026-02-20T11:43:21.240 INFO:tasks.workunit.client.0.trial072.stdout:migration_commit 2026-02-20T11:43:22.377 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.CloneV2Parent (6326 ms) 2026-02-20T11:43:22.377 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneV1ParentAbort 2026-02-20T11:43:22.619 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:43:22.727 INFO:tasks.workunit.client.0.trial072.stdout:write: A 0~10 2026-02-20T11:43:22.727 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~281s 2026-02-20T11:43:22.838 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:43:24.004 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap1 2026-02-20T11:43:24.546 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:43:26.867 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:43:28.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~286s 2026-02-20T11:43:28.895 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.CloneV1ParentAbort (6518 ms) 2026-02-20T11:43:28.896 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneV2ParentAbort 2026-02-20T11:43:29.124 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:43:29.223 INFO:tasks.workunit.client.0.trial072.stdout:write: A 0~10 2026-02-20T11:43:29.329 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:43:30.463 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap1 2026-02-20T11:43:30.947 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:43:33.279 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:43:33.533 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~292s 2026-02-20T11:43:34.886 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.CloneV2ParentAbort (5991 ms) 2026-02-20T11:43:34.887 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneV1ParentAbortFixIncompleteChildReattach 2026-02-20T11:43:35.122 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:43:35.221 INFO:tasks.workunit.client.0.trial072.stdout:write: A 0~10 2026-02-20T11:43:35.331 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:43:36.473 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap1 2026-02-20T11:43:36.977 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:43:38.936 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~297s 2026-02-20T11:43:39.515 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T11:43:40.942 INFO:tasks.workunit.client.0.trial072.stdout:[ OK ] TestMigration.CloneV1ParentAbortFixIncompleteChildReattach (6055 ms) 2026-02-20T11:43:40.943 INFO:tasks.workunit.client.0.trial072.stdout:[ RUN ] TestMigration.CloneV1ParentAbortFixParentReattach 2026-02-20T11:43:41.159 INFO:tasks.workunit.client.0.trial072.stdout:resize: 83886080 2026-02-20T11:43:41.248 INFO:tasks.workunit.client.0.trial072.stdout:write: A 0~10 2026-02-20T11:43:41.351 INFO:tasks.workunit.client.0.trial072.stdout:snap_create: snap1 2026-02-20T11:43:42.497 INFO:tasks.workunit.client.0.trial072.stdout:snap_protect: snap1 2026-02-20T11:43:43.000 INFO:tasks.workunit.client.0.trial072.stdout:migration_prepare 2026-02-20T11:43:44.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~303s 2026-02-20T11:43:44.339 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2026-02-20T11:43:44.742 INFO:tasks.ceph.osd.1:Sent signal 15 2026-02-20T11:43:44.742 INFO:tasks.ceph.osd.2:Sent signal 15 2026-02-20T11:43:44.743 INFO:tasks.ceph.mon.a:Sent signal 15 2026-02-20T11:43:44.743 INFO:tasks.ceph.mgr.x:Sent signal 15 2026-02-20T11:43:44.744 INFO:tasks.ceph.mon.a.trial072.stderr:2026-02-20T11:43:44.742+0000 7f8a447fc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 39885) UID: 0 2026-02-20T11:43:44.744 INFO:tasks.ceph.mon.a.trial072.stderr:2026-02-20T11:43:44.742+0000 7f8a447fc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-20T11:43:44.744 INFO:tasks.ceph.mgr.x.trial072.stderr:2026-02-20T11:43:44.742+0000 7f0b067f8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 39888) UID: 0 2026-02-20T11:43:44.745 INFO:tasks.ceph.mgr.x.trial072.stderr:2026-02-20T11:43:44.742+0000 7f0b067f8640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-20T11:43:45.467 INFO:tasks.workunit.client.0.trial072.stdout:migration_abort 2026-02-20T14:36:48.921 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-20T14:36:48.938 INFO:tasks.workunit:Stopping ['rbd/test_librbd.sh'] on client.0... 2026-02-20T14:36:48.938 DEBUG:teuthology.orchestra.run.trial072:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-20T14:36:49.290 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/workunit.py", line 125, in task with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/test_librbd.sh) on trial072 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=445d5c9ba18a4040e70e25303230c36c77773acb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' 2026-02-20T14:36:49.294 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-20T14:36:49.312 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T14:36:49.312 DEBUG:teuthology.orchestra.run.trial072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T14:36:49.389 INFO:teuthology.orchestra.run.trial072.stderr:2026-02-20T14:36:49.386+0000 7fe0c290b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T14:38:49.344 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-20T14:38:49.345 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-20T14:38:49.348 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-20T14:38:49.349 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-20T14:38:49.349 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-20T14:38:49.349 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T14:38:49.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T14:38:49.349 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 1525, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log' 2026-02-20T14:38:49.353 INFO:tasks.ceph.osd.0:Stopped 2026-02-20T14:38:49.353 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-20T14:38:49.353 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T14:38:49.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T14:38:49.353 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 1525, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log' 2026-02-20T14:38:49.356 INFO:tasks.ceph.osd.1:Stopped 2026-02-20T14:38:49.356 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-20T14:38:49.356 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T14:38:49.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T14:38:49.357 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 1525, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/daemon/state.py", line 146, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log' 2026-02-20T14:38:49.360 INFO:tasks.ceph.osd.2:Stopped 2026-02-20T14:38:49.360 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-20T14:38:49.360 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-20T14:38:49.360 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T14:38:49.361 INFO:tasks.ceph.mgr.x:Stopped 2026-02-20T14:38:49.361 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-20T14:38:49.361 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-20T14:38:49.361 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T14:38:49.361 INFO:tasks.ceph.mon.a:Stopped 2026-02-20T14:38:49.361 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-20T14:38:49.362 DEBUG:teuthology.orchestra.run.trial072:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v 'overall HEALTH_' | grep -E -v '\(CACHE_POOL_NO_HIT_SET\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v 'is full \(reached quota' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(OSD_' | grep -E -v '\(PG_' | grep -E -v '\(POOL_' | grep -E -v '\(CACHE_POOL_' | grep -E -v '\(POOL_FULL\)' | head -n 1 2026-02-20T14:38:49.388 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial072.front.sepia.ceph.com 2026-02-20T14:38:49.388 DEBUG:teuthology.orchestra.run.trial072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-20T14:38:49.463 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial072.front.sepia.ceph.com 2026-02-20T14:38:49.463 DEBUG:teuthology.orchestra.run.trial072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-20T14:38:49.529 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial072.front.sepia.ceph.com 2026-02-20T14:38:49.530 DEBUG:teuthology.orchestra.run.trial072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-20T14:38:49.597 INFO:tasks.ceph:Archiving mon data... 2026-02-20T14:38:49.599 DEBUG:teuthology.misc:Transferring archived files from trial072:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60522/data/mon.a.tgz 2026-02-20T14:38:49.600 DEBUG:teuthology.orchestra.run.trial072:> mktemp 2026-02-20T14:38:49.652 INFO:teuthology.orchestra.run.trial072.stdout:/tmp/tmp.6HD7zM0Mi1 2026-02-20T14:38:49.652 DEBUG:teuthology.orchestra.run.trial072:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.6HD7zM0Mi1 2026-02-20T14:38:49.753 DEBUG:teuthology.orchestra.run.trial072:> sudo chmod 0666 /tmp/tmp.6HD7zM0Mi1 2026-02-20T14:38:49.835 DEBUG:teuthology.orchestra.remote:trial072:/tmp/tmp.6HD7zM0Mi1 is 272KB 2026-02-20T14:38:49.929 DEBUG:teuthology.orchestra.run.trial072:> rm -fr /tmp/tmp.6HD7zM0Mi1 2026-02-20T14:38:49.947 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-20T14:38:49.947 DEBUG:teuthology.orchestra.run.trial072:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-20T14:38:50.067 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-20T14:38:50.067 INFO:tasks.ceph:Archiving crash dumps... 2026-02-20T14:38:50.070 DEBUG:teuthology.misc:Transferring archived files from trial072:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60522/remote/trial072/crash 2026-02-20T14:38:50.072 DEBUG:teuthology.orchestra.run.trial072:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-20T14:38:50.132 INFO:tasks.ceph:Compressing logs... 2026-02-20T14:38:50.132 DEBUG:teuthology.orchestra.run.trial072:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T14:38:50.197 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-20T14:38:50.198 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41447.log 2026-02-20T14:38:50.198 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose/var/log/ceph/ceph-osd.2.log: -- /var/log/ceph/ceph-client.admin.40742.log 2026-02-20T14:38:50.198 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41447.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-20T14:38:50.198 INFO:teuthology.orchestra.run.trial072.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41447.log.gz 2026-02-20T14:38:50.198 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39985.log 2026-02-20T14:38:50.200 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40742.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40742.log.gz 2026-02-20T14:38:50.200 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41912.log 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41001.log 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.39985.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39985.log.gz 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41912.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41912.log.gz 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41050.log 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41001.log.gz 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40837.log 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40216.log 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41050.log.gz 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40263.log 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.40837.log.gz 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40812.log 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40216.log.gz 2026-02-20T14:38:50.201 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40263.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40263.log.gz 2026-02-20T14:38:50.202 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40482.log 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40812.log: /var/log/ceph/ceph-client.admin.41300.log 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40812.log.gz 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41638.log 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40482.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40482.log.gz 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41299.log 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41300.log.gz 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40952.log 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41638.log.gz 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41299.log.gz 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40952.log.gz 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.42819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2026-02-20T14:38:50.208 INFO:teuthology.orchestra.run.trial072.stderr: 8.0% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40903.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41393.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40032.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40903.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40664.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41393.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40032.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40032.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.40664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41589.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40664.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.41589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41446.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41589.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 87.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41540.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41153.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.41446.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41152.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41540.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41153.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41153.log.gz 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39890.log 2026-02-20T14:38:50.209 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41152.log.gz 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38597.log 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.39890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41151.log 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39890.log.gz 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph.tmp-client.admin.38597.log: 66.1% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38597.log.gz 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph-client.admin.41151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40512.log 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41151.log.gz 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.40512.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40512.log.gz 2026-02-20T14:38:50.210 INFO:teuthology.orchestra.run.trial072.stderr: 87.9% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-20T14:38:50.232 INFO:teuthology.orchestra.run.trial072.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-20T14:38:50.335 INFO:teuthology.orchestra.run.trial072.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-20T14:38:51.495 INFO:teuthology.orchestra.run.trial072.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-20T14:38:55.477 INFO:teuthology.orchestra.run.trial072.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-20T14:38:55.738 INFO:teuthology.orchestra.run.trial072.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-20T14:38:55.740 INFO:teuthology.orchestra.run.trial072.stderr: 2026-02-20T14:38:55.740 INFO:teuthology.orchestra.run.trial072.stderr:real 0m5.551s 2026-02-20T14:38:55.740 INFO:teuthology.orchestra.run.trial072.stderr:user 0m11.579s 2026-02-20T14:38:55.740 INFO:teuthology.orchestra.run.trial072.stderr:sys 0m0.708s 2026-02-20T14:38:55.741 INFO:tasks.ceph:Archiving logs... 2026-02-20T14:38:55.742 DEBUG:teuthology.misc:Transferring archived files from trial072:/var/log/ceph to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60522/remote/trial072/log 2026-02-20T14:38:55.743 DEBUG:teuthology.orchestra.run.trial072:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-20T14:38:56.881 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-20T14:38:56.882 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T14:38:56.892 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/task/install/__init__.py", line 644, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-20T14:38:56.951 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T14:38:56.964 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/task/install/__init__.py", line 644, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-20T14:38:56.965 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-20T14:38:56.965 DEBUG:teuthology.orchestra.run.trial072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-20T14:38:56.991 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-20T14:38:57.002 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-20T14:38:57.002 DEBUG:teuthology.orchestra.run.trial072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T14:38:57.048 INFO:teuthology.orchestra.run.trial072.stderr:bash: line 1: ntpq: command not found 2026-02-20T14:38:57.081 INFO:teuthology.orchestra.run.trial072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T14:38:57.081 INFO:teuthology.orchestra.run.trial072.stdout:=============================================================================== 2026-02-20T14:38:57.081 INFO:teuthology.orchestra.run.trial072.stdout:^- ntp4.glypnod.com 2 10 377 22m +836us[ +823us] +/- 16ms 2026-02-20T14:38:57.081 INFO:teuthology.orchestra.run.trial072.stdout:^- time.cloudflare.com 3 8 257 157 -102us[ -102us] +/- 6479us 2026-02-20T14:38:57.081 INFO:teuthology.orchestra.run.trial072.stdout:^- ntp.sfo.icanbwell.com 2 10 377 126 -1187us[-1187us] +/- 36ms 2026-02-20T14:38:57.081 INFO:teuthology.orchestra.run.trial072.stdout:^* pool-138-89-14-60.mad.ea> 1 10 377 565 -47us[ -61us] +/- 3203us 2026-02-20T14:38:57.083 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-20T14:38:57.094 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-20T14:38:57.094 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-20T14:38:57.107 DEBUG:teuthology.orchestra.run.trial072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-20T14:38:57.177 DEBUG:teuthology.orchestra.run.trial072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T14:38:57.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T14:38:57.203 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-20T14:38:57.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-20T14:38:57.229 INFO:teuthology.task.internal:Duration was 11713.675060 seconds 2026-02-20T14:38:57.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-20T14:38:57.246 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-20T14:38:57.246 DEBUG:teuthology.orchestra.run.trial072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-20T14:38:57.284 INFO:teuthology.orchestra.run.trial072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T14:38:57.441 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-20T14:38:57.441 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial072.front.sepia.ceph.com 2026-02-20T14:38:57.441 DEBUG:teuthology.orchestra.run.trial072:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-20T14:38:57.501 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-20T14:38:57.501 DEBUG:teuthology.orchestra.run.trial072:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T14:38:57.606 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-20T14:38:57.606 DEBUG:teuthology.orchestra.run.trial072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T14:38:57.628 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T14:38:57.628 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T14:38:57.628 INFO:teuthology.orchestra.run.trial072.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-20T14:38:57.628 INFO:teuthology.orchestra.run.trial072.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T14:38:57.629 INFO:teuthology.orchestra.run.trial072.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-20T14:38:57.639 INFO:teuthology.orchestra.run.trial072.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 84.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-20T14:38:57.641 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-20T14:38:57.653 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-20T14:38:57.653 DEBUG:teuthology.orchestra.run.trial072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-20T14:38:57.705 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-20T14:38:57.718 DEBUG:teuthology.orchestra.run.trial072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-20T14:38:57.768 INFO:teuthology.orchestra.run.trial072.stdout:kernel.core_pattern = core 2026-02-20T14:38:57.799 DEBUG:teuthology.orchestra.run.trial072:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-20T14:38:57.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T14:38:57.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-20T14:38:57.875 INFO:teuthology.task.internal:Transferring archived files... 2026-02-20T14:38:57.876 DEBUG:teuthology.misc:Transferring archived files from trial072:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60522/remote/trial072 2026-02-20T14:38:57.876 DEBUG:teuthology.orchestra.run.trial072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-20T14:38:57.934 INFO:teuthology.task.internal:Removing archive directory... 2026-02-20T14:38:57.935 DEBUG:teuthology.orchestra.run.trial072:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-20T14:38:57.984 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-20T14:38:57.995 INFO:teuthology.task.internal:Not uploading archives. 2026-02-20T14:38:57.996 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-20T14:38:58.010 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-20T14:38:58.010 DEBUG:teuthology.orchestra.run.trial072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-20T14:38:58.041 INFO:teuthology.orchestra.run.trial072.stdout: 917576 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 20 14:38 /home/ubuntu/cephtest 2026-02-20T14:38:58.041 INFO:teuthology.orchestra.run.trial072.stdout: 1311123 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 20 11:36 /home/ubuntu/cephtest/mnt.0 2026-02-20T14:38:58.042 INFO:teuthology.orchestra.run.trial072.stdout: 1311124 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 20 11:36 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-20T14:38:58.042 INFO:teuthology.orchestra.run.trial072.stdout: 1327064 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 11:36 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-20T14:38:58.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T14:38:58.042 INFO:teuthology.orchestra.run.trial072.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-20T14:38:58.042 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph.py", line 2010, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2936, in wait_for_clean num_active_clean = self.get_num_active_clean() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2715, in get_num_active_clean pgs = self.get_pg_stats() ^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 2481, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1713, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks/ceph_manager.py", line 1704, in run_cluster_cmd return self.controller.run(**kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial072 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-20T14:38:58.044 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-20T14:38:58.055 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-20T14:38:58.069 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial072 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2026-02-20T14:38:58.075 INFO:teuthology.run:Summary data: description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/rbd_api_tests} duration: 11713.675059556961 failure_reason: 'Command failed (workunit test rbd/test_librbd.sh) on trial072 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=445d5c9ba18a4040e70e25303230c36c77773acb TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh''' flavor: debug owner: scheduled_jjperez@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-20T14:38:58.075 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T14:38:58.205 INFO:teuthology.run:FAIL